Getting Data From Json Into Mongo Using Fluentd
Looking to get data out of json into mongo? You can do that with fluentd in 10 minutes!
Here is how:
$ gem install fluentd $ gem install fluent-plugin-mongo $ touch fluentd.conf
fluentd.conf should look like this (just copy and paste this into fluentd.conf):
<source> @type tail path /var/log/httpd-access.log #...or where you placed your Apache access log pos_file /var/log/td-agent/httpd-access.log.pos # This is where you record file position tag foobar.json #fluentd tag! format json # one JSON per line time_key time_field # optional; default = time </source> <match **> @type mongo database <db name> #(required) collection <collection name> #(optional; default="untagged") host <hostname> #(optional; default="localhost") port <port> #(optional; default=27017) </match>
After that, you can start fluentd and everything should work:
$ fluentd -c fluentd.conf
Of course, this is just a quick example. If you are thinking of running fluentd in production, consider using td-agent, the enterprise version of Fluentd packaged and maintained by Treasure Data, Inc..
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF), originally invented by Treasure Data, Inc. All components are available under the Apache 2 License.