This article shows how to use Fluentd to collect GlusterFS logs for analysis (search, analytics, troubleshooting, etc.)
Background
​GlusterFS is an open source, distributed file system commercially supported by Red Hat, Inc. Each node in GlusterFS generates its own logs, and it's sometimes convenient to have these logs collected in a central location for analysis (e.g., When one GlusterFS node went down, what was happening on other nodes?).
​Fluentd is an open source data collector for high-volume data streams. It's a great fit for monitoring GlusterFS clusters because:
1.
Fluentd supports GlusterFS logs as a data source.
2.
Fluentd supports various output systems (e.g., Elasticsearch,
MongoDB, Treasure Data, etc.) that can help GlusterFS users analyze
the logs.
The rest of this article explains how to set up Fluentd with GlusterFS. For this example, we chose Elasticsearch as the backend system.
Setting up Fluentd on GlusterFS Nodes
Step 1: Installing Fluentd
First, we'll install Fluentd using the following command:
1
$ curl -L https://toolbelt.treasuredata.com/sh/install-redhat-td-agent2.sh | sh
Copied!
Next, we'll install the Fluentd plugin for GlusterFS:
$ ls -alF /var/log/glusterfs/etc-glusterfs-glusterd.vol.log
5
-rw-r--r-- 1 root root 1385 Feb 3 07:21 2014 /var/log/glusterfs/etc-glusterfs-glusterd.vol.log
Copied!
Now, modify Fluentd's configuration file. It is located at /etc/td-agent/td-agent.conf. `td-agent` is Fluentd's rpm/deb package maintained by Treasure Data​
This is what the configuration file should look like: