Recipe Http Rest Api To Treasure Data

Looking to get data out of http rest api into treasure data? You can do that with fluentd in 10 minutes!
Here is how:
1
$ gem install fluentd
2
$ gem install fluent-plugin-td
3
$ touch fluentd.conf
Copied!
fluentd.conf should look like this (just copy and paste this into fluentd.conf):
1
<source>
2
  @type http
3
  port 8888
4
  bind 0.0.0.0
5
  body_size_limit 32m
6
  keepalive_timeout 10s
7
  # tag is part of the URL, e.g.,
8
  # curl -X POST -d 'json={"action":"login","user":2}' http://localhost:8888/tag.here
9
</source>
10
​
11
<match **>
12
  @type tdlog
13
  apikey <Treasure Data API key> # You get your API key by signing up for Treasure Data
14
  auto_create_table
15
  buffer_type file
16
  buffer_path /var/log/td-agent/buffer/td
17
</match>
Copied!
After that, you can start fluentd and everything should work:
1
$ fluentd -c fluentd.conf
Copied!
Of course, this is just a quick example. If you are thinking of running fluentd in production, consider using td-agent, the enterprise version of Fluentd packaged and maintained by Treasure Data, Inc..
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.

Recipe Http Rest Api To S3

Recipe Json To Elasticsearch

Last modified 2yr ago

Copy link