Fluentd
Search…
Python
The fluent-logger-python library is used to post records from Python applications to Fluentd.
This article explains how to use it.

Prerequisites

    Basic knowledge of Python
    Basic knowledge of Fluentd
    Python 2.6 or higher

Installing Fluentd

Please refer to the following documents to install fluentd:

Modifying the Config File

Configure Fluentd to use the forward input plugin as its data source:
1
<source>
2
@type forward
3
port 24224
4
</source>
5
<match fluentd.test.**>
6
@type stdout
7
</match>
Copied!
Restart agent after configuring.
1
# for rpm/deb only
2
$ sudo /etc/init.d/td-agent restart
3
4
# or systemd
5
$ sudo systemctl restart td-agent.service
Copied!

Using fluent-logger-python

Install fluent-logger library via pip:
1
$ pip install fluent-logger
Copied!
Initialize and post the records as shown below:
1
# test.py
2
from fluent import sender
3
from fluent import event
4
sender.setup('fluentd.test', host='localhost', port=24224)
5
event.Event('follow', {
6
'from': 'userA',
7
'to': 'userB'
8
})
Copied!
Executing the script will send the logs to Fluentd:
1
$ python test.py
Copied!
The logs should be output to /var/log/td-agent/td-agent.log or the standard output of the Fluentd process via the stdout output plugin.

Production Deployments

Output Plugins

Various output plugins are available for writing records to other destinations:

High-Availability Configurations of Fluentd

For high-traffic websites (more than 5 application nodes), we recommend using the high-availability configuration for td-agent. This will improve the reliability of data transfer and query performance.

Monitoring

Monitoring Fluentd itself is also important. The article below describes the general monitoring methods for td-agent.
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is an open-source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.
Last modified 4mo ago