Versions | v0.14 (td-agent3) | v0.12 (td-agent2)

This page is for v0.14, not the latest stable version which is v0.12. For the latest stable version of this article, click here.


grep Filter Plugin

The filter_grep filter plugin “greps” events by the values of specified fields.

Table of Contents

Example Configurations

filter_grep is included in Fluentd’s core. No installation required.

<filter foo.bar>
  @type grep
  regexp1 message cool
  regexp2 hostname ^web\d+\.example\.com$
  exclude1 message uncool
</filter>

The above example matches any event that satisfies the following conditions:

  1. The value of the “message” field contains “cool”
  2. The value of the “hostname” field matches web<INTEGER>.example.com.
  3. The value of the “message” field does NOT contain “uncool”.

Hence, the following events are kept:

{"message":"It's cool outside today", "hostname":"web001.example.com"}
{"message":"That's not cool", "hostname":"web1337.example.com"}

whereas the following examples are filtered out:

{"message":"I am cool but you are uncool", "hostname":"db001.example.com"}
{"hostname":"web001.example.com"}
{"message":"It's cool outside today"}

Parameters

regexpN (optional)

The “N” at the end should be replaced with an integer between 1 and 20 (ex: “regexp1”). regexpN takes two whitespace-delimited arguments.

  • The first argument is the field name to which the regular expression is applied.
  • The second argument is the regular expression.

For example, the following filters out events unless the field “price” is a positive integer.

regexp1 price [1-9]\d*

The grep filter filters out UNLESS all regexpN’s are matched. Hence, if you have

regexp1 price [1-9]\d*
regexp2 item_name ^book_

unless the event’s “item_name” field starts with “book_” and the “price” field is an integer, it is filtered out.

For OR condition, you can use | operator of regular expressions. For example, if you have

regexp1 item_name (^book_|^article)

unless the event’s “item_name” field starts with “book” or “article”, it is filtered out.

Learn regular expressions for more patterns.

excludeN (optional)

The “N” at the end should be replaced with an integer between 1 and 20 (ex: “exclude1”). excludeN takes two whitespace-delimited arguments.

  • The first argument is the field name to which the regular expression is applied.
  • The second argument is the regular expression.

For example, the following filters out events whose “status_code” field is 5xx.

exclude1 status_code ^5\d\d$

The grep filter filters out if any excludeN is matched. Hence, if you have

exclude1 status_code ^5\d\d$
exclude2 url \.css$

Then, any event whose “status_code” is 5xx OR “url” ends with “.css” is filtered out.

If regexpN and excludeN are used together, both are applied.

Learn More

Last updated: 2016-09-21 02:58:49 UTC

Versions | v0.14 (td-agent3) | v0.12 (td-agent2)

If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF), originally invented by Treasure Data, Inc. All components are available under the Apache 2 License.

Interested in the Fluentd Newsletters?