Send Syslog Data to Graylog
This article explains how to set up Fluentd with Graylog. Graylog is a popular log management server powered by Elasticsearch and MongoDB. You can combine Fluentd and Graylog to create a scalable log analytics pipeline.
Prerequisites
Basic Understanding of Fluentd
Linux Server (Ubuntu 18.04 LTS was used for this guide.)
How to Setup Graylog + Fluentd
Dependencies
Install the dependencies with the following command:
Elasticsearch
Graylog requires Elasticsearch, which can be instantly launched using the following commands:
Elasticsearch is ready. Start it with:
Graylog
In this article, we will use Graylog 2.4.
Update the package cache and install graylog-server
:
Open /etc/graylog/server/server.conf
and configure the following parameters:
password_secret
root_password_sha2
.web_enable
For root_password_sha2
, run echo -n ROOT_PASSWORD | sha256sum
and set the hash. Also you need to set web_enable
to true to access the web interface.
Now let's start Graylog:
Prepare Graylog for Fluentd
Go to http://localhost:9000 and log into the web interface.
To log in, use admin
as the username and YOUR_PASSWORD
as the password (the one you have set up for root_password_sha2
).
Once logged in, click on System
in the top nav. Next, click on Inputs
from the left navigation bar. (Or, simply go to http://localhost:9000/system/inputs.
Then, from the dropdown, choose GELF UDP
and click on Launch new input
, which should pop up a modal dialogue, Select the Node
and fill the Title
. Then, click Save
.
Now, Graylog2 is ready to accept messages from Fluentd over UDP. It is time to configure Fluentd.
Fluentd
See the download page for all the options. Here, we are using the deb package:
Then, install the out_gelf
plugin to send data to Graylog. Currently, the GELF plugin is not available on RubyGems, so we need to download the plugin file and place it in /etc/td-agent/plugin
:
We also need to gem-install GELF's Ruby client:
Configure /etc/td-agent/td-agent.conf
as follows:
Open /etc/rsyslog.conf
and add the following line to the file:
Finally, restart rsyslog
and Fluentd with the following commands:
Visualize the Data Stream
When you log back into Graylog, you should be seeing a graph like this (wait for events to flow in):
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is an open-source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.
Last updated