Routing Examples

This article shows typical routing examples.
Simple Input -> Filter -> Output
<source>
  @type forward
</source>
​
<filter app.**>
  @type record_transformer
  <record>
    hostname "#{Socket.gethostname}"
  </record>
</filter>
​
<match app.**>
  @type file
  # ...
</match>
Two input cases
<source>
  @type forward
</source>
​
<source>
  @type tail
  tag system.logs
  # ...
</source>
​
<filter app.**>
  @type record_transformer
  <record>
    hostname "#{Socket.gethostname}"
  </record>
</filter>
​
<match {app.**,system.logs}>
  @type file
  # ...
</match>
If you want to separate data pipeline for each sources, use Label.
Input -> Filter -> Output with Label
Label reduces complex tag handling by separating data pipeline.
<source>
  @type forward
</source>
​
<source>
  @type dstat
  @label @METRICS # dstat events are routed to <label @METRICS>
  # ...
</source>
​
<filter app.**>
  @type record_transformer
  <record>
    # ...
  </record>
</filter>
​
<match app.**>
  @type file
  # ...
</match>
​
<label @METRICS>
  <match **>
    @type elasticsearch
    # ...
  </match>
</label>
Re-route event by tag
Use fluent-plugin-route plugin. route plugin rewrites tag and re-emit events to other match or Label.
<match worker.**>
  @type route
  remove_tag_prefix worker
  add_tag_prefix metrics.event
​
  <route **>
    copy # For fall-through. Without copy, routing is stopped here. 
  </route>
  <route **>
    copy
    @label @BACKUP
  </route>
</match>
​
<match metrics.event.**>
  @type stdout
</match>
​
<label @BACKUP>
  <match metrics.event.**>
    @type file
    path /var/log/fluent/backup
  </match>
</label>
Re-route event by record content
Use fluent-plugin-rewrite-tag-filter.
<source>
  @type forward
</source>
​
# event example: app.logs {"message":"[info]: ..."}
<match app.**>
  @type rewrite_tag_filter
  <rule>
    key message
    pattern ^\[(\w+)\]
    tag $1.${tag}
  </rule>
  # you can put more <rule>
</match>
​
# send mail when receives alert level logs
<match alert.app.**>
  @type mail
  # ...
</match>
​
# other logs are stored into file
<match *.app.**>
  @type file
  # ...
</match>
See also out_rewrite_tag_filter article.
Re-route event to other Label
Use out_relabel plugin. relabel plugin simply emits events to Label. No tag rewrite.
<source>
  @type forward
</source>
​
<match app.**>
  @type copy
  <store>
    @type forward
    # ...
  </store>
  <store>
    @type relabel
    @label @NOTIFICATION
  </store>
</match>
​
<label @NOTIFICATION>
  <filter app.**>
    @type grep
    regexp1 message ERROR
  </filter>
​
  <match app.**>
    @type mail
  </match>
</label>
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.
Config File Syntax
Config: Common Parameters
Last updated 5 months ago