You can also change the logging level with <system> section in the config file like below.
<system>
# equal to -qq option
log_level error
</system>
Per Plugin Log
The log_level option sets different levels of logging for each plugin. It can be set in each plugin's configuration file.
For example, in order to debug in_tail but suppress all but fatal log messages for in_http, their respective log_level options should be set as follows:
Same stacktrace is replaced with suppressed same stacktrace message until other stacktrace is received.
Output to log file
Fluentd outputs logs to STDOUT by default. To output to a file instead, please specify the -o option.
$ fluentd -o /path/to/log_file
Fluentd doesn't support log rotation yet.
Capture Fluentd logs
Fluentd marks its own logs with the fluent tag. You can process Fluentd logs by using <match fluent.**> or <match **>(Of course, ** captures other logs). If you define <match fluent.**> in your configuration, then Fluentd will send its own logs to this match destination. This is useful for monitoring Fluentd logs.
For example, if you have the following <match fluent.**>:
# omit other source / match
<match fluent.**>
@type stdout
</match>
then Fluentd outputs fluent.info logs to stdout like below:
2014-02-27 00:00:00 +0900 [info]: shutting down fluentd
2014-02-27 00:00:01 +0900 fluent.info: {"message":"shutting down fluentd"} # by <match fluent.**>
2014-02-27 00:00:01 +0900 [info]: process finished code = 0
Case1: Send Fluentd logs to monitoring service
You can send Fluentd logs to a monitoring service by plugins, e.g. datadog, sentry, irc, etc.
# Add hostname for identifying the server
<filter fluent.**>
@type record_transformer
<record>
host "#{Socket.gethostname}"
</record>
</filter>
<match fluent.**>
@type monitoring_plugin
# parameters...
</match>
Case2: Use aggregation/monitoring server
You can use out_forward to send Fluentd logs to a monitoring server. The monitoring server can then filter and send the logs to your notification system: chat, irc, etc.
Leaf server example:
# Add hostname for identifying the server and tag to filter by log level
<filter fluent.**>
@type record_transformer
<record>
host "#{Socket.gethostname}"
original_tag ${tag}
</record>
</filter>
<match fluent.**>
@type forward
<server>
# Monitoring server parameters
</server>
</match>
Monitoring server example:
<source>
@type forward
label @FLUENTD_INTERNAL_LOG
</source>
<label @FLUENTD_INTERNAL_LOG>
# Ignore trace, debug and info log
<filter fluent.**>
@type grep
regexp1 original_tag fluent.(warn|error|fatal)
</filter>
<match fluent.**>
# your notification setup. This example uses irc plugin
@type irc
host irc.domain
channel notify
message notice: %s [%s] @%s %s
out_keys original_tag,time,host,message
</match>
</label>
If an error occurs, you will get a notification message in your irc notify channel.
If you don't specify the log_level parameter, the plugin will use the global log level. Some plugins haven't supported per-plugin logging yet. The explains how to update such plugins to support the new log level system.
If this article is incorrect or outdated, or omits critical information, please . is a open source project under . All components are available under the Apache 2 License.
