regexp

The regexp parser plugin parses logs by given regexp pattern. The regexp must have at least one named capture (?<NAME>PATTERN). If the regexp has a capture named time, this is configurable via time_key parameter, it is used as the time of the event. You can specify the time format using the time_format parameter.

<parse>
@type regexp
expression /.../
</parse>

Parameters

See Parse section configurations for common parameters.

expression

type

default

version

regexp

required parameter

1.2.0

Regular expression for matching logs. Regular expression also supports i and m suffix.

i (ignorecase)

Ignore case in matching.

expression /.../i

m (multiline)

Build regular expression as a multiline mode. . matches newline. See Ruby's Regexp document

expression /.../m

both

expression /.../im

expression is string type before 1.2.0.

ignorecase

type

default

version

bool

false

0.14.2

Ignore case in matching. Use i option with expression.

Deprecated since 1.2.0. Use expression /pattern/i instead.

multiline

type

default

version

bool

false

0.14.2

Build regular expression as a multline mode. . matches newline. See Ruby's Regexp document Use m option with expression.

Deprecated since 1.2.0. Use expression /pattern/m instead.

Example

<parse>
@type regexp
expression /^\[(?<logtime>[^\]]*)\] (?<name>[^ ]*) (?<title>[^ ]*) (?<id>\d*)$/
time_key logtime
time_format %Y-%m-%d %H:%M:%S %z
types id:integer
</parse>

With this config:

[2013-02-28 12:00:00 +0900] alice engineer 1

This incoming log is parsed as:

time:
1362020400 (2013-02-28 12:00:00 +0900)
record:
{
"name" : "alice",
"title": "engineer",
"id" : 1
}

FAQ

How to debug my regexp pattern?

fluentd-ui's in_tail editor helps your regexp testing. Another way, Fluentular is a great website to test your regexp for Fluentd configuration.

NOTE: You may hit Application Error at Fluentular due to heroku free plan limitation. Retry a few hours later or use fluentd-ui instead.

If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.