flush_interval(e.g. 1s) with smaller
buffer_chunk_size(e.g. 1MB) to immediately forward incoming data to 2nd tier output processes.
in_tailcan now focus on tailing and parsing, rather than sharing the CPU power for filtering and output.
out_forwardautomatically distribute the load across the processes automatically.
td-agent, or directly use Ruby to launch Fluentd.
in_multiprocessto launch both input and output processes.
./tmpl/directory has the config generation script, in case you'd like to increase the processes.
./prometheusdirectory, which contains example
in_xyz.conf. Please make sure you have
<match>section, to forward incoming data to 2nd tier output processes. Finally, register the process in
out_N.conf, and register the process in both
fluentd.confwhich list up all the output processes.