<buffer>
section (See the diagram below).<buffer>
sections in the configuration. If the users specify <buffer>
section for the output plugins that do not support buffering, Fluentd will raise configuration errors.<match>
section for both the configuration parameters of output and buffer plugins. Fluentd v1.0 uses <buffer>
subsection to write parameters for buffering, flushing and retrying. <match>
sections are used only for the output plugin itself.flush_mode
default
default
, lazy
, interval
, immediate
interval
flushes per flush_interval
immediate
flushes just after event arrivesflush_interval
60
flush_thread_count
1
flush_thread_interval
1.0
flush_thread_burst_interval
1.0
delayed_commit_timeout
60
slow_flush_log_threshold
20.0
(seconds)float
, not time
.overflow_action
throw_exception
(default)BufferOverflowError
exception to the input plugin. How BufferOverflowError
is handled depends on the input plugins, e.g. tail input stops reading new lines, forward input returns an error to forward output. This action is suitable for streaming.block
in_tail
plugin. Other input plugins, e.g. socket-based plugin, don't assume this action.block
action to avoid BufferOverflowError
. Please consider improving destination settings to resolve BufferOverflowError
or use @ERROR
label for routing overflowed events to another backup destination (or secondary
with lower retry_limit
). If you hit BufferOverflowError
frequently, it means your destination capacity is insufficient for your traffic.drop_oldest_chunk
retry_wait
). If the retry limit has not been disabled (retry_forever
is false
) and the retry count exceeds the specified limit (retry_max_times
), all chunks in the queue are discarded. The retry wait time doubles each time (1.0sec, 2.0sec, 4.0sec, ...) until retry_max_interval
is reached. If the queue length exceeds the specified limit (queue_limit_length
), new events are rejected.Output#write
or Output#try_write
method throws an exception.retry_timeout: 100s
.retry_type
exponential_backoff
(default)periodic
retry_forever
true
, plugin will ignore retry_timeout
and retry_max_times
options and retry flushing forever.false
retry_timeout
72h
(72 hours)retry_max_times
retry_timeout
is the default, the number is 18 with exponential backoff.nil
retry_secondary_threshold
retry_timeout
to switch to use secondary while failing.0.8
retry_wait
1
retry_exponential_backoff_base
2
retry_max_interval
nil
retry_randomize
true
, the output plugin will retry after a randomized interval not to do burst retries.true
buffered
mode, the user can specify <secondary>
with any output plugin in <match>
configuration. If plugins continue to fail writing buffer chunks and exceeds the timeout threshold for retries, then output plugins will delegate the writing of the buffer chunk to the secondary plugin.<secondary>
is useful for backup when destination servers are unavailable, e.g. forward
, mongo
, etc. We strongly recommend out_secondary_file
plugin for <secondary>
./var/log/td-agent/buffer/elasticsearch
, and any failure will be sent to /var/log/td-agent/error/
using my.logs
for file names:<secondary>
plugin receives the primary's buffer chunk directly. So, you need to check if your secondary plugin works with the primary setting.retry_timeout: 100s
with the secondary.retry_max_times: 10
with the secondary.