Versions | v1.0 (td-agent3)

Compat Parameters Plugin Helper API

compat_parameters helper convert parameters from v0.12 style to v1.0 style.

Here is the code example with compat_parameters helper:

require 'fluent/plugin/output'

module Fluent::Plugin
  class ExampleOutput < Output
    Fluent::Plugin.register_output('example', self)

    # 1. load compat_parameters helper
    helpers :compat_parameters

    # omit start, shutdown and other plugin API

    def configure(conf)
      compat_parameters_convert(conf, :buffer, :inject, default_chunk_key: "time")
      super
      ...
    end
  end
end

Table of Contents

Methods

compat_parameters_convert(conf, *types, **kwargs)

  • conf: Fluent::Configuration instance
  • types: Following 5 types are supported
    • :buffer
    • :inject
    • :extract
    • :parser
    • :formatter
  • kwargs:

The important point is you can’t mix v1.0 and v0.12 styles in one plugin directive. If you mix v1.0 and v0.12 styles, v1.0 style is used and v0.12 style is ignored. Here is an example:

<match pattern>
  @type foo
  # This flush_interval is ignored because <buffer> exist. Conversion doesn't happen.
  flush_interval 30s
  # This <buffer> parameters are used
  <buffer>
    @type file
    path /path/to/buffer
    retry_max_times 10
    queue_limit_length 256
  </buffer>
</match>

buffer

old (v0.12) new (v1) note
buffer_type @type
buffer_path path
num_threads flush_thread_count
flush_interval flush_interval
try_flush_interval flush_thread_interval
queued_chunk_flush_interval flush_thread_burst_interval
disable_retry_limit retry_forever
retry_limit retry_max_times
max_retry_wait retry_max_interval
buffer_chunk_limit chunk_limit_size
buffer_queue_limit queue_limit_length
buffer_queue_full_action overflow_action
flush_at_shutdown flush_at_shutdown
time_slice_format timekey also set chunk_key to ‘time’
time_slice_wait timekey_wait
timezone timekey_zone
localtime timekey_use_utc exclusive with utc
utc timekey_use_utc exclusive with localtime

This converts following flat configuration:

buffer_type file
buffer_path /path/to/buffer
retry_limit 10
flush_interval 30s
buffer_queue_limit 256

into:

<buffer>
  @type file
  path /path/to/buffer
  retry_max_times 10
  flush_interval 30s
  queue_limit_length 256
</buffer>

For more details, see Buffer section configuration

inject

old (v0.12) new (v1) note
include_time_key time_key if true, set time_key
time_key time_key
time_format time_format
timezone timezone
include_tag_key tag_key if true, set tag_key
tag_key tag_key
localtime localtime exclusive with utc
utc localtime exclusive with localtime

This converts following flat configuration:

include_time_key
time_key event_time
utc

into:

<inject>
  time_key event_time
  localtime false
</inject>

For more details, see Inject Plugin Helper API

extract

old (v0.12) new (v1) note
time_key time_key
time_format time_format
timezone timezone
tag_key tag_key
localtime localtime exclusive with utc
utc localtime exclusive with localtime

This converts following flat configuration:

include_time_key
time_key event_time
utc

into:

<extract>
  time_key event_time
  localtime false
</extract>

For more details, see Extract Plugin Helper API

parser

old (v0.12) new (v1) note plugin
format @type
types types converted to JSON format
types_delimiter types
types_label_delimiter types
keys keys CSVParser, TSVParser (old ValuesParser)
time_key time_key
time_format time_format
localtime localtime exclusive with utc
utc localtime exclusive with localtime
delimiter delimiter
keep_time_key keep_time_key
null_empty_string null_empty_string
null_value_pattern null_value_pattern
json_parser json_parser JSONParser
label_delimiter label_delimiter LabeledTSVParser
format_firstline format_firstline MultilineParser
message_key message_key NoneParser
with_priority with_priority SyslogParser
message_format message_format SyslogParser
rfc5424_time_format rfc5424_time_format SyslogParser

This converts following configuration:

format /^(?<log_time>[^ ]*) (?<message>)+*/
time_key log_time
time_format %Y%m%d%H%M%S
keep_time_key

into:

<parse>
  @type regexp
  pattern /^(?<log_time>[^ ]*) (?<message>)+*/
  time_key log_time
  time_format %Y%m%d%H%M%S
  keep_time_key
</parse>

For more details, see Parser Plugin Overview and Writing Parser Plugins.

formatter

old (v0.12) new (v1) note plugin
format @type
delimiter delimiter
force_quotes force_quotes CsvFormatter
keys keys TSVFormatter
fields fields CsvFormatter
json_parser json_parser JSONFormatter
label_delimiter label_delimiter LabeledTSVFormatter
output_time output_time OutFileFormatter
output_tag output_tag OutFileFormatter
localtime localtime exclusive with utc OutFileFormatter
utc utc exclusive with localtime OutFileFormatter
timezone timezone OutFileFormatter
message_key message_key SingleValueFormatter
add_newline add_newline SingleValueFormatter
output_type output_type StdoutFormatter

This converts following configuration:

format json

into:

<format>
  @type json
</format>

For more details, see Formatter Plugin Overview and Writing Formatter Plugins.

compat_parameters used plugins

Last updated: 2018-08-20 22:07:07 +0000

Versions | v1.0 (td-agent3)

If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.