Fluentd
Search…
0.12
Introduction
Overview
Use Cases
Configuration
Deployment
Container Deployment
Input Plugins
Output Plugins
Buffer Plugins
Filter Plugins
Parser Plugins
Formatter Plugins
Developer
Articles
Store Apache Logs into MongoDB
Apache To Riak
Store Apache Logs into Amazon S3
Before Install
Cep Norikra
Collect Glusterfs Logs
Common Log Formats
Docker Logging Efk Compose
Docker Logging
Filter Modify Apache
Forwarding Over Ssl
Free Alternative To Splunk By Fluentd
Data Collection to Hadoop (HDFS)
Data Analytics with Treasure Data
Install By Chef
Install By Deb
Install By Dmg
Install By Gem
Install By Rpm
Install From Source
Install On Beanstalk
Install On Heroku
Java
Kinesis Stream
Kubernetes Fluentd
Monitoring Prometheus
Monitoring Rest Api
Nodejs
Performance Tuning Multi Process
Performance Tuning Single Process
Perl
Php
Python
Quickstart
Raspberrypi Cloud Data Logger
Recipe Apache Logs To Elasticsearch
Recipe Apache Logs To Mongo
Recipe Apache Logs To S3
Recipe Apache Logs To Treasure Data
Recipe Cloudstack To Mongodb
Recipe Csv To Elasticsearch
Recipe Csv To Mongo
Recipe Csv To S3
Recipe Csv To Treasure Data
Recipe Http Rest Api To Elasticsearch
Recipe Http Rest Api To Mongo
Recipe Http Rest Api To S3
Recipe Http Rest Api To Treasure Data
Recipe Json To Elasticsearch
Recipe Json To Mongo
Recipe Json To S3
Recipe Json To Treasure Data
Recipe Nginx To Elasticsearch
Recipe Nginx To Mongo
Recipe Nginx To S3
Recipe Nginx To Treasure Data
Recipe Syslog To Elasticsearch
Recipe Syslog To Mongo
Recipe Syslog To S3
Recipe Syslog To Treasure Data
Recipe Tsv To Elasticsearch
Recipe Tsv To Mongo
Recipe Tsv To S3
Recipe Tsv To Treasure Data
Ruby
Scala
Splunk Like Grep And Alert Email
Powered By
GitBook
Python
The '
fluent-logger-python
', library is used to post records from Python applications to Fluentd.
This article explains how to use the fluent-logger-python library.
Prerequisites
Basic knowledge of Python
Basic knowledge of Fluentd
Python 2.6 or higher
Installing Fluentd
Please refer to the following documents to install fluentd.
​
Install Fluentd with rpm Package
​
​
Install Fluentd with deb Package
​
​
Install Fluentd with Ruby Gem
​
​
Install Fluentd from source
​
Modifying the Config File
Next, please configure Fluentd to use the
forward Input plugin
as its data source.
1
<source>
2
@type forward
3
port 24224
4
</source>
5
<match fluentd.test.**>
6
@type stdout
7
</match>
Copied!
Please restart your agent once these lines are in place.
1
# for rpm/deb only
2
$ sudo /etc/init.d/td-agent restart
Copied!
Using fluent-logger-python
First, install the fluent-logger library via pip.
1
$ pip install fluent-logger
Copied!
Next, initialize and post the records as shown below.
1
# test.py
2
from fluent import sender
3
from fluent import event
4
sender.setup('fluentd.test', host='localhost', port=24224)
5
event.Event('follow', {
6
'from': 'userA',
7
'to': 'userB'
8
})
Copied!
Executing the script will send the logs to Fluentd.
1
$ python test.py
Copied!
The logs should be output to
/var/log/td-agent/td-agent.log
or stdout of the Fluentd process via the
stdout Output plugin
.
Production Deployments
Output Plugins
Various
output plugins
are available for writing records to other destinations:
Examples
​
Store Apache Logs into Amazon S3
​
​
Store Apache Logs into MongoDB
​
​
Data Collection into HDFS
​
List of Plugin References
​
Output to Another Fluentd
​
​
Output to MongoDB
or
MongoDB ReplicaSet
​
​
Output to Hadoop
​
​
Output to File
​
​
etc...
​
High-Availability Configurations of Fluentd
For high-traffic websites (more than 5 application nodes), we recommend using a high availability configuration of td-agent. This will improve data transfer reliability and query performance.
​
High-Availability Configurations of Fluentd
​
Monitoring
Monitoring Fluentd itself is also important. The article below describes general monitoring methods for td-agent.
​
Monitoring Fluentd
​
If this article is incorrect or outdated, or omits critical information, please
let us know
.
Fluentd
is a open source project under
Cloud Native Computing Foundation (CNCF)
. All components are available under the Apache 2 License.
Previous
Php
Next
Quickstart
Last modified
2yr ago
Copy link
Contents
Prerequisites
Installing Fluentd
Modifying the Config File
Using fluent-logger-python
Production Deployments
Output Plugins
High-Availability Configurations of Fluentd
Monitoring