Python

The 'fluent-logger-python', library is used to post records from Python applications to Fluentd.
This article explains how to use the fluent-logger-python library.
Prerequisites
Basic knowledge of Python
Basic knowledge of Fluentd
Python 2.6 or higher
Installing Fluentd
Please refer to the following documents to install fluentd.
​Install Fluentd with rpm Package​
​Install Fluentd with deb Package​
​Install Fluentd with Ruby Gem​
​Install Fluentd from source​
Modifying the Config File
Next, please configure Fluentd to use the forward Input plugin as its data source.
1
<source>
2
  @type forward
3
  port 24224
4
</source>
5
<match fluentd.test.**>
6
  @type stdout
7
</match>
Copied!
Please restart your agent once these lines are in place.
1
# for rpm/deb only
2
$ sudo /etc/init.d/td-agent restart
Copied!
Using fluent-logger-python
First, install the fluent-logger library via pip.
1
$ pip install fluent-logger
Copied!
Next, initialize and post the records as shown below.
1
# test.py
2
from fluent import sender
3
from fluent import event
4
sender.setup('fluentd.test', host='localhost', port=24224)
5
event.Event('follow', {
6
  'from': 'userA',
7
  'to':   'userB'
8
})
Copied!
Executing the script will send the logs to Fluentd.
1
$ python test.py
Copied!
The logs should be output to /var/log/td-agent/td-agent.log or stdout of the Fluentd process via the stdout Output plugin.
Production Deployments
Output Plugins
Various output plugins are available for writing records to other destinations:
Examples
​Store Apache Logs into Amazon S3​
​Store Apache Logs into MongoDB​
​Data Collection into HDFS​
List of Plugin References
​Output to Another Fluentd​
​Output to MongoDB or MongoDB ReplicaSet​
​Output to Hadoop​
​Output to File​
​etc...​
High-Availability Configurations of Fluentd
For high-traffic websites (more than 5 application nodes), we recommend using a high availability configuration of td-agent. This will improve data transfer reliability and query performance.
​High-Availability Configurations of Fluentd​
Monitoring
Monitoring Fluentd itself is also important. The article below describes general monitoring methods for td-agent.
​Monitoring Fluentd​
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.
Php
Quickstart
Last modified 2yr ago
Copy link
Contents
Prerequisites
Installing Fluentd
Modifying the Config File
Using fluent-logger-python
Production Deployments
Output Plugins
High-Availability Configurations of Fluentd
Monitoring