syslog
is that services have a wide range of log formats, and no single parser can parse all syslog
messages effectively.syslog
messages robustly.rsyslogd
rsyslogd
/etc/rsyslogd.conf
and append the following line:rsyslogd
service:rsyslogd
to forward logs to port 5140 to which Fluentd will listen.syslog
Messages/etc/td-agent/td-agent.conf
and put the following configuration:td-agent
:syslog
Messages from sudo
sudo
message like this one:sudo
. In order to do so, we need to parse the message field. In other words, we need to extract syslog
messages from sudo
and handle them differently.grep
filter plugin. It examines the fields of events, and filter them based on regular expression patterns. In the following example, Fluentd filters out events that come from sudo
and contain command data:syslog
messages. For this purpose, we use another plugin called filter-parser
. With this plugin, you can parse the content of a field using a regular expression.td-agent
:sudo
:/var/log/td-agent/td-agent.log
:syslog
events. You can immediately send data to the output systems like MongoDB and Elasticsearch, but also you can do filtering and further parsing inside Fluentd before passing the processed data onto the output destinations.