Fluentd
Search…
Python
The fluent-logger-python library is used to post records from Python applications to Fluentd.
This article explains how to use it.

Prerequisites

  • Basic knowledge of Python
  • Basic knowledge of Fluentd
  • Python 2.6 or higher

Installing Fluentd

Please refer to the following documents to install fluentd:

Modifying the Config File

Configure Fluentd to use the forward input plugin as its data source:
1
<source>
2
@type forward
3
port 24224
4
</source>
5
<match fluentd.test.**>
6
@type stdout
7
</match>
Copied!
Restart agent after configuring.
1
# for rpm/deb only
2
$ sudo /etc/init.d/td-agent restart
3
4
# or systemd
5
$ sudo systemctl restart td-agent.service
Copied!

Using fluent-logger-python

Install fluent-logger library via pip:
1
$ pip install fluent-logger
Copied!
Initialize and post the records as shown below:
1
# test.py
2
from fluent import sender
3
from fluent import event
4
sender.setup('fluentd.test', host='localhost', port=24224)
5
event.Event('follow', {
6
'from': 'userA',
7
'to': 'userB'
8
})
Copied!
Executing the script will send the logs to Fluentd:
1
$ python test.py
Copied!
The logs should be output to /var/log/td-agent/td-agent.log or the standard output of the Fluentd process via the stdout output plugin.

Production Deployments

Output Plugins

Various output plugins are available for writing records to other destinations:

High-Availability Configurations of Fluentd

For high-traffic websites (more than 5 application nodes), we recommend using the high-availability configuration for td-agent. This will improve the reliability of data transfer and query performance.

Monitoring

Monitoring Fluentd itself is also important. The article below describes the general monitoring methods for td-agent.
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is an open-source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.
Last modified 1yr ago