Post Installation Guide

The goal of this article is to provide a concise post-installation guide to new Fluentd users. It is assumed that you've installed Fluentd through td-agent package.

System Administration

Configuration File

A clean installation leaves you a td-agent instance running on a sample configuration file. You can edit the configuration file located at:


After editing this file, you need to restart td-agent using systemctl:

$ sudo systemctl restart td-agent


By default, td-agent writes its operation logs to the following file:


If you want to make td-agent more verbose, read the article "Trouble Shooting".

Connect to Other Services

How It Works

In Fluentd, the most important part of data input/output is managed by plugins. Each plugin knows how to interface with a external endpoint and is responsible for managing a pipeline to convey data streams.

Plugins are named with a certain convention. For example, if it receives data and interfacing with Apache Kafka, it's called in_kafka. In the same way, if it publishes data and connects to MongoDB, it's called out_mongo.

The following snippet is an example configuration, which uses in_forward plugin as an input source and out_file plugin as an output endpoint.

@type forward
port 9999
<match app.**>
@type file
path /var/log/app/data.log
compress gzip

Plugin Management

Fluentd manages plugins as Ruby gems, but stores these gems in a separate directory from where normal Ruby gems reside.

This is why you need to use a special program td-agent-gem to manage Fluentd plugins. For example, the following command allows you to install the plugin to connect S3 (which contains both in_s3 and out_s3)

$ sudo /usr/sbin/td-agent-gem install fluent-plugin-s3

Available Plugins

See List Of All Plugins to explore available third-party plugins.

Note that a number of plugins are already included in the standard distribution of td-agent, so you may not need to install them manually.

Configuration Syntax

Data Source

A configuration file consists of a number of setting blocks (like <source>). Each block contains a set of options for a specific data endpoint.

For example, if you want to create an endpoint to receive data from syslog, you need to add a <source> block and set up its settings as follows.

@type syslog
port 5140
tag system

The option @type determines which plugin to use. You do not need prepend type prefix in this option (so @type syslog, not @type in_syslog).

Output Endpoint

To add an output endpoint for data stream, you need to define a <match> block. Syntactically, <match> is slightly different from <source> in the sense that it requires a filter expression as an argument.

For example, If you want to output events tagged with debug.log, you need to write as below:

<match debug.log>
@type kafka2
port 5140
brokers kafka-server:9092
tag system
# other parameters...

You can use a wildcard character * in the filter expression. For example, debug.* matches debug.log and etc.

If you want to catch all descendent tags, use double asterisks **. For example, debug.** matches not only debug.log, but also or debug.log.level.critical etc.

Further Reading

Read Configuration File Syntax for the full configuration syntax.

If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.