Parser Plugins

Fluentd has eight (8) types of plugins:

This article gives an overview of Parser Plugin.

Overview

Sometimes, the <parse> directive for input plugins (e.g. in_tail, in_syslog, in_tcp and in_udp) cannot parse the user's custom data format (for example, a context-dependent grammar that can't be parsed with a regular expression). To address such cases, Fluentd has a pluggable system that enables the user to create their own parser formats.

How To Use

  • Write a custom format plugin. See here

    for more information.

  • From any input plugin that supports the <parse> directive, call the custom

    plugin by its name.

Here is a simple example to read Nginx access logs using in_tail and parser_nginx:

<source>
@type tail
path /path/to/input/file
<parse>
@type nginx
keep_time_key true
</parse>
</source>

List of Built-in Parsers

Third-party Parsers

  • grok

    If you are familiar with grok patterns, grok-parser plugin is useful. Use > 1.0.0 versions for fluentd v0.14/v1.0.

  • multi-format-parser

    If you need to parse multiple formats in one data stream, multi-format-parser is useful.

  • protobuf

    For protocol buffers.

List of Core Input Plugins with Parser support

Following plugins support <parse> directive:

If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is an open-source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.