Fluentd
0.12
0.12
  • Introduction
  • Overview
    • Getting Started
    • Installation
    • Life of a Fluentd event
    • Support
    • FAQ
  • Use Cases
    • Centralized App Logging
    • Monitoring Service Logs
    • Data Analytics
    • Connecting to Data Storages
    • Stream Processing
    • Windows Event Collection
    • IoT Data Logger
  • Configuration
    • Config File Syntax
    • Routing Examples
    • Recipes
  • Deployment
    • Logging
    • Monitoring
    • Signals
    • RPC
    • High Availability Config
    • Failure Scenarios
    • Performance Tuning
    • Plugin Management
    • Trouble Shooting
    • Secure Forwarding
    • Fluentd UI
    • Command Line Option
  • Container Deployment
    • Docker Image
    • Docker Logging Driver
    • Docker Compose
    • Kubernetes
  • Input Plugins
    • tail
    • forward
    • secure_forward
    • udp
    • tcp
    • http
    • unix
    • syslog
    • exec
    • scribe
    • multiprocess
    • dummy
    • Others
  • Output Plugins
    • file
    • s3
    • kafka
    • forward
    • secure_forward
    • exec
    • exec_filter
    • copy
    • geoip
    • roundrobin
    • stdout
    • null
    • webhdfs
    • splunk
    • mongo
    • mongo_replset
    • relabel
    • rewrite_tag_filter
    • Others
  • Buffer Plugins
    • memory
    • file
  • Filter Plugins
    • record_transformer
    • grep
    • parser
    • stdout
  • Parser Plugins
    • regexp
    • apache2
    • apache_error
    • nginx
    • syslog
    • ltsv
    • csv
    • tsv
    • json
    • multiline
    • none
  • Formatter Plugins
    • out_file
    • json
    • ltsv
    • csv
    • msgpack
    • hash
    • single_value
  • Developer
    • Plugin Development
    • Community
    • Mailing List
    • Source Code
    • Bug Tracking
    • ChangeLog
    • Logo
  • Articles
    • Store Apache Logs into MongoDB
    • Apache To Riak
    • Store Apache Logs into Amazon S3
    • Before Install
    • Cep Norikra
    • Collect Glusterfs Logs
    • Common Log Formats
    • Docker Logging Efk Compose
    • Docker Logging
    • Filter Modify Apache
    • Forwarding Over Ssl
    • Free Alternative To Splunk By Fluentd
    • Data Collection to Hadoop (HDFS)
    • Data Analytics with Treasure Data
    • Install By Chef
    • Install By Deb
    • Install By Dmg
    • Install By Gem
    • Install By Rpm
    • Install From Source
    • Install On Beanstalk
    • Install On Heroku
    • Java
    • Kinesis Stream
    • Kubernetes Fluentd
    • Monitoring by Prometheus
    • Monitoring by Rest Api
    • Nodejs
    • Performance Tuning Multi Process
    • Performance Tuning Single Process
    • Perl
    • Php
    • Python
    • Quickstart
    • Raspberrypi Cloud Data Logger
    • Recipe Apache Logs To Elasticsearch
    • Recipe Apache Logs To Mongo
    • Recipe Apache Logs To S3
    • Recipe Apache Logs To Treasure Data
    • Recipe Cloudstack To Mongodb
    • Recipe Csv To Elasticsearch
    • Recipe Csv To Mongo
    • Recipe Csv To S3
    • Recipe Csv To Treasure Data
    • Recipe Http Rest Api To Elasticsearch
    • Recipe Http Rest Api To Mongo
    • Recipe Http Rest Api To S3
    • Recipe Http Rest Api To Treasure Data
    • Recipe Json To Elasticsearch
    • Recipe Json To Mongo
    • Recipe Json To S3
    • Recipe Json To Treasure Data
    • Recipe Nginx To Elasticsearch
    • Recipe Nginx To Mongo
    • Recipe Nginx To S3
    • Recipe Nginx To Treasure Data
    • Recipe Syslog To Elasticsearch
    • Recipe Syslog To Mongo
    • Recipe Syslog To S3
    • Recipe Syslog To Treasure Data
    • Recipe Tsv To Elasticsearch
    • Recipe Tsv To Mongo
    • Recipe Tsv To S3
    • Recipe Tsv To Treasure Data
    • Ruby
    • Scala
    • Splunk Like Grep And Alert Email
Powered by GitBook
On this page
  • What is td-agent?
  • Step 0: Before Installation
  • Step 1 : Install from Apt Repository
  • Step2: Launch Daemon
  • Step3: Post Sample Logs via HTTP
  • Next Steps

Was this helpful?

  1. Articles

Install By Deb

This article explains how to install the td-agent deb package, the stable Fluentd distribution package maintained by Treasure Data, Inc.

What is td-agent?

Fluentd is written in Ruby for flexibility, with performance sensitive parts written in C. However, some users may have difficulty installing and operating a Ruby daemon.

That's why Treasure Data, Inc is providing the stable community distribution of Fluentd, called td-agent. The differences between Fluentd and td-agent can be found here.

Currently, td-agent 2 deb has two versions, td-agent 2.5 / td-agent 2.3. The different point is bundled ruby version. td-agent 2.5 or later uses ruby 2.5 and td-agent 2.3 or earlier uses ruby 2.1. ruby 2.1 is EOL so we recommend to use td-agent 2.5 for new deployment. td-agent 2.5 and td-agent 2.3 use fluentd v0.12 serise so the behaviour is same.

Step 0: Before Installation

Please follow the Preinstallation Guide to configure your OS properly. This will prevent many unnecessary problems.

Step 1 : Install from Apt Repository

For Ubuntu, we currently support "Ubuntu 18.04 LTS / Bionic 64bit", "Ubuntu 16.04 LTS / Xenial 64bit", "Ubuntu 14.04 LTS / Trusty 64bit/32bit", "Ubuntu 12.04 LTS / Precise 64bit/32bit", "Ubuntu 10.04 LTS / Lucid 64bit/32bit".

For Debian, we currently support Jessie, Wheezy and Squeeze. There is no 32bit packages for Debian. We might not provide td-agent for older distributions. For example, we don't provide latest td-agent version for squeeze and lucid for now.

A shell script is provided to automate the installation process for each version. The shell script registers a new apt repository at /etc/apt/sources.list.d/treasure-data.list and installs the td-agent deb package.

For Bionic,

curl -L https://toolbelt.treasuredata.com/sh/install-ubuntu-bionic-td-agent2.5.sh | sh

For Xenial,

# td-agent 2.5 or later
curl -L https://toolbelt.treasuredata.com/sh/install-ubuntu-xenial-td-agent2.5.sh | sh
# td-agent 2.3 or earlier
curl -L https://toolbelt.treasuredata.com/sh/install-ubuntu-xenial-td-agent2.sh | sh

For Trusty,

# td-agent 2.5 or later
curl -L https://toolbelt.treasuredata.com/sh/install-ubuntu-trusty-td-agent2.5.sh | sh
# td-agent 2.3 or earlier
curl -L https://toolbelt.treasuredata.com/sh/install-ubuntu-trusty-td-agent2.sh | sh

For Precise,

curl -L https://toolbelt.treasuredata.com/sh/install-ubuntu-precise-td-agent2.sh | sh

For Lucid,

curl -L https://toolbelt.treasuredata.com/sh/install-ubuntu-lucid-td-agent2.sh | sh

For Debian Stretch,

curl -L https://toolbelt.treasuredata.com/sh/install-debian-stretch-td-agent2.sh | sh

For Debian Jessie,

curl -L https://toolbelt.treasuredata.com/sh/install-debian-jessie-td-agent2.sh | sh

For Debian Wheezy,

curl -L https://toolbelt.treasuredata.com/sh/install-debian-wheezy-td-agent2.sh | sh

For Debian Squeeze,

curl -L https://toolbelt.treasuredata.com/sh/install-debian-squeeze-td-agent2.sh | sh

It's HIGHLY recommended that you set up ntpd on the node to prevent invalid timestamps in your logs. Please check the Preinstallation Guide.

Step2: Launch Daemon

The /etc/init.d/td-agent script is provided to start, stop, or restart the agent.

$ sudo /etc/init.d/td-agent restart
$ sudo /etc/init.d/td-agent status
td-agent (pid  21678) is running...

The following commands are supported:

$ sudo /etc/init.d/td-agent start
$ sudo /etc/init.d/td-agent stop
$ sudo /etc/init.d/td-agent restart
$ sudo /etc/init.d/td-agent status

Please make sure your configuration file is located at /etc/td-agent/td-agent.conf.

Step3: Post Sample Logs via HTTP

By default, /etc/td-agent/td-agent.conf is configured to take logs from HTTP and route them to stdout (/var/log/td-agent/td-agent.log). You can post sample log records using the curl command.

$ curl -X POST -d 'json={"json":"message"}' http://localhost:8888/debug.test

Next Steps

You're now ready to collect your real logs using Fluentd. Please see the following tutorials to learn how to collect your data from various data sources.

  • Basic Configuration

    • Config File

  • Application Logs

    • Ruby, Java, Python, PHP,

      Perl, Node.js, Scala

  • Examples

    • Store Apache Log into Amazon S3

    • Store Apache Log into MongoDB

    • Data Collection into HDFS

Please refer to the resources below for further steps.

  • ChangeLog of td-agent

  • Chef Cookbook

If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.

PreviousInstall By ChefNextInstall By Dmg

Last updated 4 years ago

Was this helpful?