Docker Image

This article explains how to use Fluentd's official Docker image, maintained by Treasure Data, Inc.

• Fluentd's official Docker image

• Fluentd's official Docker image (Source)

Step 0: Install Docker

Please download and install Docker from here.

• Docker Installation

Step 1: Pull Fluentd's Docker image

Then, please download Fluentd v0.12's image by docker pull command.

$ docker pull fluent/fluentd:v0.12-debian

Debian and Alpine Linux version is available for Fluentd image. Debian version is recommended officially since it has jemalloc support, however Alpine image is smaller.

Step 2: Launch Fluentd Container

To make the test simple, create the example config below at /tmp/fluentd.conf. This example accepts records from http, and output to stdout.

# /tmp/fluentd.conf
<source>
  @type http
  port 9880
  bind 0.0.0.0
</source>
<match **>
  @type stdout
</match>

Finally, you can run Fluentd with docker run command.

$ docker run -d \
  -p 9880:9880 -v /tmp:/fluentd/etc -e FLUENTD_CONF=fluentd.conf \
  fluent/fluentd
2017-01-30 11:52:23 +0000 [info]: reading config file path="/fluentd/etc/fluentd.conf"
2017-01-30 11:52:23 +0000 [info]: starting fluentd-0.12.31
2017-01-30 11:52:23 +0000 [info]: gem 'fluentd' version '0.12.31'
2017-01-30 11:52:23 +0000 [info]: adding match pattern="**" type="stdout"
2017-01-30 11:52:23 +0000 [info]: adding source type="http"
2017-01-30 11:52:23 +0000 [info]: using configuration file: <ROOT>
  <source>
    @type http
    port 9880
    bind 0.0.0.0
  </source>
  <match **>
    @type stdout
  </match>
</ROOT>

Step3: Post Sample Logs via HTTP

Let's post sample logs via HTTP and confirm it's working. curl command is always your friend.

$ curl -X POST -d 'json={"json":"message"}' http://localhost:9880/sample.test

Use docker ps command to retrieve container ID, and use docker logs command to check the specific container's log.

$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                         NAMES
b495e527850c        fluent/fluentd      "/bin/sh -c 'exec ..."   2 hours ago         Up 2 hours          5140/tcp, 24224/tcp, 0.0.0.0:9880->9880/tcp   awesome_mcnulty
$ docker logs b495e527850c | tail -n 1
2017-01-30 14:04:37 +0000 sample.test: {"json":"message"}

Next Steps

Now you know how to use Fluentd via Docker. Here're a couple of Docker related documentations for Fluentd.

• Fluentd's official Docker image

• Fluentd's official Docker image (Source)

• Docker Logging Driver and Fluentd

• Docker Logging via EFK (Elasticsearch + Fluentd + Kibana) Stack with Docker Compose

Also, please see the following tutorials to learn how to collect your data from various data sources.

• Basic Configuration

  • Config File

• Application Logs

  • Ruby, Java, Python, PHP,

    Perl, Node.js, Scala

• Examples

  • Store Apache Log into Amazon S3

  • Store Apache Log into MongoDB

  • Data Collection into HDFS

If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.