Fluentd
Search…
0.12
Install On Heroku
This article describes how to install Fluentd (td-agent) on Heroku.

With non-Treasure Data backend

Heroku doesn't allow users to install separate processes within a single dyno. You will thus need to setup Fluentd as a separate Heroku application. This will become you central log aggregation server.
Treasure Data provides a boilerplate repository to get you started. Follow the steps below to create Fluentd (td-agent) as a Heroku application.
1
# Clone
2
$ git clone git://github.com/treasure-data/heroku-td-agent.git
3
$ cd heroku-td-agent
4
$ rm -fR .git
5
$ git init
6
$ git add .
7
$ git commit -m 'initial commit'
8
9
# Create the app & deploy
10
$ heroku create --stack cedar
11
$ git push heroku master
12
13
# Update your config
14
$ vi td-agent.conf
15
$ git commit -a -m 'update config file'
16
17
# Deploy
18
$ git push heroku master
Copied!

With Treasure Data as a Backend

Since Treasure Data, the primary sponsor of Fluentd, is a Heroku Addons provider, some of you came here to stream data to Treasure Data through Fluentd.
If
1
http
2
Your App ------> Heroku-hosted Fluentd ------> Treasure Data
Copied!
is what you want, there are two ways to deploy it.

1. Heroku Button

Clicking on the following button automatically launched td-agent on Heroku:
You can read more about this approach here.

2. Using Git

This is similar to "With non-Treasure Data Backend", except you do not need to edit the config. Instead, simply update the configuration variable:
1
# Clone
2
$ git clone git://github.com/treasure-data/heroku-td-agent.git
3
$ cd heroku-td-agent
4
$ rm -fR .git
5
$ git init
6
$ git add .
7
$ git commit -m 'initial commit'
8
9
# Create the app & deploy
10
$ heroku create --stack cedar
11
$ git push heroku master
12
13
# Set your config
14
$ heroku config:set TREASURE_DATA_API_KEY=you_treasure_data_api_key_value
15
16
# Deploy
17
$ git push heroku master
Copied!
To retrieve your treasure_data_api_key_value, see here.

Test

Let's confirm that the log aggregation server has been set up correctly. Please send a GET request to the log server, http://td-agent-on-heroku.herokuapp.com, as shown below. This will log the event { "json": "message" } with a debug.sample tag. Note how the JSON-formatted data is passed as a query parameter value.
1
$ curl "http://td-agent-on-heroku.herokuapp.com/debug.sample?json=%7B%22json%22%3A%22message%22%7D"
Copied!
In general, the URL format is
1
http://{YOUR LOG SERVER DOMAIN}/td.{DB_NAME}.{TABLE_NAME}?json={JSON_FORMATTED_DATA}
Copied!
The output will be available on STDOUT.
1
$ heroku logs --tail
Copied!
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.