Plugin Helper: Compat Parameters

compat_parameters helper convert parameters from v0.12 style to v1.0 style.

Here is the code example with compat_parameters helper:

require 'fluent/plugin/output'
module Fluent::Plugin
class ExampleOutput < Output
Fluent::Plugin.register_output('example', self)
# 1. load compat_parameters helper
helpers :compat_parameters
# omit start, shutdown and other plugin API
def configure(conf)
compat_parameters_convert(conf, :buffer, :inject, default_chunk_key: "time")
super
...
end
end
end

Methods

compat_parameters_convert(conf, *types, **kwargs)

  • conf: Fluent::Configuration instance

  • types: Following 5 types are supported

    • :buffer

    • :inject

    • :extract

    • :parser

    • :formatter

  • kwargs:

The important point is you can't mix v1.0 and v0.12 styles in one plugin directive. If you mix v1.0 and v0.12 styles, v1.0 style is used and v0.12 style is ignored. Here is an example:

<match pattern>
@type foo
# This flush_interval is ignored because <buffer> exist. Conversion doesn't happen.
flush_interval 30s
# This <buffer> parameters are used
<buffer>
@type file
path /path/to/buffer
retry_max_times 10
queue_limit_length 256
</buffer>
</match>

buffer

old (v0.12)

new (v1)

note

buffer_type

@type

buffer_path

path

num_threads

flush_thread_count

num_threads

flush_thread_count

flush_interval

flush_interval

try_flush_interval

flush_thread_interval

queued_chunk_flush_interval

flush_thread_burst_interval

disable_retry_limit

retry_forever

retry_limit

retry_max_times

max_retry_wait

retry_max_interval

buffer_chunk_limit

chunk_limit_size

buffer_queue_limit

queue_limit_length

buffer_queue_full_action

overflow_action

flush_at_shutdown

flush_at_shutdown

time_slice_format

timekey

also set chunk_key to 'time'

time_slice_wait

timekey_wait

timezone

timekey_zone

localtime

timekey_use_utc

exclusive with utc

utc

timekey_use_utc

exclusive with localtime

This converts following flat configuration:

buffer_type file
buffer_path /path/to/buffer
retry_limit 10
flush_interval 30s
buffer_queue_limit 256

into:

<buffer>
@type file
path /path/to/buffer
retry_max_times 10
flush_interval 30s
queue_limit_length 256
</buffer>

For more details, see Buffer section configuration

inject

old (v0.12)

new (v1)

note

include_time_key

time_key

if true, set time_key

time_key

time_key

time_format

time_format

timezone

timezone

include_tag_key

tag_key

if true, set tag_key

tag_key

tag_key

localtime

localtime

exclusive with utc

utc

localtime

exclusive with localtime

This converts following flat configuration:

include_time_key
time_key event_time
utc

into:

<inject>
time_key event_time
localtime false
</inject>

For more details, see Inject Plugin Helper API

extract

old (v0.12)

new (v1)

note

time_key

time_key

time_format

time_format

timezone

timezone

tag_key

tag_key

localtime

localtime

exclusive with utc

utc

localtime

exclusive with localtime

This converts following flat configuration:

include_time_key
time_key event_time
utc

into:

<extract>
time_key event_time
localtime false
</extract>

For more details, see Extract Plugin Helper API

parser

old (v0.12)

new (v1)

note

plugin

format

@type

types

types

converted to JSON format

types_delimiter

types

types_label_delimiter

types

keys

keys

CSVParser, TSVParser (old ValuesParser)

time_key

time_key

time_format

time_format

localtime

localtime

exclusive with utc

utc

localtime

exclusive with localtime

delimiter

delimiter

keep_time_key

keep_time_key

null_empty_string

null_empty_string

null_value_pattern

null_value_pattern

json_parser

json_parser

JSONParser

label_delimiter

label_delimiter

LabeledTSVParser

format_firstline

format_firstline

MultilineParser

message_key

message_key

NoneParser

with_priority

with_priority

SyslogParser

message_format

message_format

SyslogParser

rfc5424_time_format

rfc5424_time_format

SyslogParser

This converts following configuration:

format /^(?<log_time>[^ ]*) (?<message>)+*/
time_key log_time
time_format %Y%m%d%H%M%S
keep_time_key

into:

<parse>
@type regexp
pattern /^(?<log_time>[^ ]*) (?<message>)+*/
time_key log_time
time_format %Y%m%d%H%M%S
keep_time_key
</parse>

For more details, see Parser Plugin Overview and Writing Parser Plugins.

formatter

old (v0.12)

new (v1)

note

plugin

format

@type

delimiter

delimiter

force_quotes

force_quotes

CsvFormatter

keys

keys

TSVFormatter

fields

fields

CsvFormatter

json_parser

json_parser

JSONFormatter

label_delimiter

label_delimiter

LabeledTSVFormatter

output_time

output_time

OutFileFormatter

output_tag

output_tag

OutFileFormatter

localtime

localtime

exclusive with utc

OutFileFormatter

utc

utc

exclusive with localtime

OutFileFormatter

timezone

timezone

OutFileFormatter

message_key

message_key

SingleValueFormatter

add_newline

add_newline

SingleValueFormatter

output_type

output_type

StdoutFormatter

This converts following configuration:

format json

into:

<format>
@type json
</format>

For more details, see Formatter Plugin Overview and Writing Formatter Plugins.

compat_parameters used plugins

If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.