Recipe Nginx To Elasticsearch

Looking to get data out of nginx into elasticsearch? You can do that with fluentd in 10 minutes!
Here is how:
1
$ gem install fluentd
2
$ gem install fluent-plugin-elasticsearch
3
$ touch fluentd.conf
Copied!
fluentd.conf should look like this (just copy and paste this into fluentd.conf):
1
<source>
2
  @type tail
3
  path /var/log/httpd-access.log #...or where you placed your Apache access log
4
  pos_file /var/log/td-agent/httpd-access.log.pos # This is where you record file position
5
  tag nginx.access #fluentd tag!
6
  format nginx # Do you have a custom format? You can write your own regex.
7
</source>
8
​
9
<match **>
10
  @type elasticsearch
11
  logstash_format true
12
  host <hostname> #(optional; default="localhost")
13
  port <port> #(optional; default=9200)
14
  index_name <index name> #(optional; default=fluentd)
15
  type_name <type name> #(optional; default=fluentd)
16
</match>
Copied!
After that, you can start fluentd and everything should work:
1
$ fluentd -c fluentd.conf
Copied!
Of course, this is just a quick example. If you are thinking of running fluentd in production, consider using td-agent, the enterprise version of Fluentd packaged and maintained by Treasure Data, Inc..
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.

Recipe Json To Treasure Data

Recipe Nginx To Mongo

Last modified 2yr ago

Copy link