Fluentd
Search…
0.12
Powered By GitBook
Common Log Formats
This page is a glossary of common log formats that can be parsed with the Tail input plugin.
    Apache Access Log
    Use format apache2 as shown below:
    1
    <source>
    2
    @type tail
    3
    format apache2
    4
    tag apache.access
    5
    path /var/log/apache2/access.log
    6
    </source>
    Copied!
    Apache Error Log
    Use a regular expression. See the format field in the following sample configuration.
    1
    <source>
    2
    @type tail
    3
    format /^\[[^ ]* (?<time>[^\]]*)\] \[(?<level>[^\]]*)\] \[pid (?<pid>[^\]]*)\] \[client (?<client>[^\]]*)\] (?<message>.*)$/
    4
    tag apache.error
    5
    path /var/log/apache2/error.log
    6
    </source>
    Copied!
    Depending on your particular error log format, you may need to adjust the regular expression above. You can test your format using fluentd-ui's in_tail editor or Fluentular.
    Maillog
    Use a regular expression. See the format field in the following sample configuration.
    1
    <source>
    2
    @type tail
    3
    format /^(?<time>[^ ]+) (?<host>[^ ]+) (?<process>[^:]+): (?<message>((?<key>[^ :]+)[ :])? ?((to|from)=<(?<address>[^>]+)>)?.*)$/
    4
    tag postfix.maillog
    5
    path /var/log/maillog
    6
    </source>
    Copied!
    Nginx Access Log
    Use format nginx as shown below:
    1
    <source>
    2
    @type tail
    3
    format nginx
    4
    tag nginx.access
    5
    path /var/log/nginx/access.log
    6
    </source>
    Copied!
    Nginx Error Log
    Use the format* and multiline_flush_interval fields in the following sample configuration. Applications running under Nginx can output multi-line errors including stack traces, so the multiline mode is a good fit.
    1
    <source>
    2
    @type tail
    3
    tag nginx.error
    4
    path /var/log/nginx/error.log
    5
    6
    format multiline
    7
    format_firstline /^\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2} \[\w+\] (?<pid>\d+).(?<tid>\d+): /
    8
    format1 /^(?<time>\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}) \[(?<log_level>\w+)\] (?<pid>\d+).(?<tid>\d+): (?<message>.*)/
    9
    multiline_flush_interval 3s
    10
    </source>
    Copied!
    If you know your error log will only contain single lines, you can use the below simpler configuration with just a format.
    1
    <source>
    2
    @type tail
    3
    format /^(?<time>\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}) \[(?<log_level>\w+)\] (?<pid>\d+).(?<tid>\d+): (?<message>.*)$/
    4
    tag nginx.error
    5
    path /var/log/nginx/error.log
    6
    </source>
    Copied!
    GlusterFS Logs

Do you not see what you are looking for?

Give us a shout on GitHub, Twitter or the mailing list. Better yet, we always welcome a pull request!
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.
Last modified 2yr ago