The filter_grep
filter plugin "greps" events by the values of specified fields.
It is included in the Fluentd's core.
<filter foo.bar>@type grep​<regexp>key messagepattern /cool/</regexp>​<regexp>key hostnamepattern /^web\d+\.example\.com$/</regexp>​<exclude>key messagepattern /uncool/</exclude></filter>
The above example matches any event that satisfies the following conditions:
The value of the message
field contains cool
.
The value of the hostname
field matches web<INTEGER>.example.com
.
The value of the message
field does NOT contain uncool
.
Hence, the following events are kept:
{"message":"It's cool outside today", "hostname":"web001.example.com"}{"message":"That's not cool", "hostname":"web1337.example.com"}
whereas the following examples are filtered out:
{"message":"I am cool but you are uncool", "hostname":"db001.example.com"}{"hostname":"web001.example.com"}{"message":"It's cool outside today"}
​record_accessor
​
​Common Parameters​
Specifies the filtering rule. This directive contains either <regexp>
or <exclude>
directive. This directive has been added since 1.2.0.
<and><regexp>key pricepattern /[1-9]\d*/</regexp>​<regexp>key item_namepattern /^book_/</regexp></and>
This is same as below:
<regexp>key pricepattern /[1-9]\d*/</regexp>​<regexp>key item_namepattern /^book_/</regexp>
We can also use <and>
directive with <exclude>
directive:
<and><exclude>key container_namepattern /^app\d{2}/</exclude>​<exclude>key log_levelpattern /^(?:debug|trace)$/</exclude></and>
Specifies the filtering rule. This directive contains either <regexp>
or <exclude>
directive. This directive has been added since 1.2.0.
<or><exclude>key status_codepattern /^5\d\d$/</exclude>​<exclude>key urlpattern /\.css$/</exclude></or>
This is same as below:
<exclude>key status_codepattern /^5\d\d$/</exclude>​<exclude>key urlpattern /\.css$/</exclude>
We can also use <or>
directive with <regexp>
directive:
<or><regexp>key container_namepattern /^db\d{2}/</regexp>​<regexp>key log_levelpattern /^(?:warn|error)$/</regexp></or>
Specifies the filtering rule. This directive contains two parameters:
key
pattern
type | default | version |
string | required parameter | 1.0.0 |
The field name to which the regular expression is applied.
This parameter supports nested field access via record_accessor
syntax.
type | default | version |
regexp | required parameter | 1.2.0 |
The regular expression.
The pattern parameter is string type before 1.2.0.
For example, the following filters out events unless the field price
is a positive integer.
<regexp>key pricepattern /[1-9]\d*/</regexp>
The grep
filter filters out UNLESS all <regexp>
s are matched. Hence, if you have:
<regexp>key pricepattern /[1-9]\d*/</regexp>​<regexp>key item_namepattern /^book_/</regexp>
unless the event's item_name
field starts with book_
and the price
field is an integer, it is filtered out.
For OR condition, you can use |
operator of regular expressions. For example, if you have:
<regexp>key item_namepattern /(^book_|^article)/</regexp>
unless the event's item_name
field starts with book*
or article*
, it is filtered out.
Note that if you want to use a match pattern with a leading slash (a typical case is a file path), you need to escape the leading slash. Otherwise, the pattern will not be recognized as expected.
Here is a simple example:
<regexp>key filepathpattern \/spool/</regexp>
You can also write the pattern like this:
<regexp>key filepathpattern /\/spool\//</regexp>
Learn regular expressions for more patterns.
type | version |
string | 1.0.0 |
This is a deprecated parameter. Use <regexp>
instead.
The N
at the end should be replaced with an integer between 1 and 20 (e.g. regexp1
). regexpN
takes two whitespace-delimited arguments.
Here is regexpN
version of <regexp>
example:
regexp1 price [1-9]\d*regexp2 item_name ^book_
Specifies the filtering rule to reject events. This directive contains two parameters:
key
pattern
type | default | version |
string | required parameter | 1.0.0 |
The field name to which the regular expression is applied.
This parameter supports nested field access via record_accessor
syntax.
type | default | version |
regexp | required parameter | 1.2.0 |
The regular expression.
The pattern parameter is string type before 1.2.0.
For example, the following filters out events whose status_code
field is 5xx:
<exclude>key status_codepattern /^5\d\d$/</exclude>
The grep
filter filters out if any <exclude>
is matched. Hence, if you have:
<exclude>key status_codepattern /^5\d\d$/</exclude>​<exclude>key urlpattern /\.css$/</exclude>
Then, any event with status_code
of 5xx
OR url
ending with .css
is filtered out.
type | version |
string | 1.0.0 |
This is a deprecated parameter. Use <exclude>
instead.
The N
at the end should be replaced with an integer between 1 and 20 (e.g. exclude1
). excludeN
takes two whitespace-delimited arguments.
Here is excludeN
version of <exclude>
example:
exclude1 status_code ^5\d\d$exclude2 url \.css$
If <regexp>
and <exclude>
are used together, both are applied.
​Filter Plugin Overview​
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is an open-source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.