Versions | v0.14 (td-agent3) | v0.12 (td-agent2) | v0.10 (td-agent1)

This page is for v0.14, not the latest stable version which is v0.12. For the latest stable version of this article, click here.


Parser Plugin Overview

Fluentd has 6 types of plugins: Input, Parser, Filter, Output, Formatter, Storage and Buffer. This article gives an overview of Parser Plugin.

Table of Contents

Overview

Sometimes, the <parse> directive for input plugins (ex: in_tail, in_syslog, in_tcp and in_udp) cannot parse the user’s custom data format (for example, a context-dependent grammar that can’t be parsed with a regular expression). To address such cases. Fluentd has a pluggable system that enables the user to create their own parser formats.

How To Use

  • Write a custom format plugin. See here for more information.
  • From any input plugin that supports the <parse> directive, call the custom plugin by its name.

Here is an example with in_tail:

<source>
  @type tail
  path /path/to/input/file
  <parse>
    @type my_custom_parser
  </parse>
</source>

List of Built-in Parsers

3rd party Parsers

If you are familiar with grok patterns, grok-parser plugin is useful. Use > 1.0.0 versions for fluentd v0.14.

List of Core Input Plugins with Parser support

with <parse> directive.

Last updated: 2016-06-13 06:11:23 UTC

Versions | v0.14 (td-agent3) | v0.12 (td-agent2) | v0.10 (td-agent1)

If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.

Interested in the Fluentd Newsletters?