multi-worker
feature.top
Commandtop
command first. Try to identify which part of your system is becoming a bottleneck (CPU, Memory, Disk I/O, etc.).flush_thread_count
Parameterflush_thread_count
option will parallelize your outputs (the default is 1). Using multiple threads can hide the IO/network latency. This parameter is available for all output plugins.gzip
Command for S3/TDgzip
. This frees up the Ruby interpreter while allowing Fluentd to process other tasks.flush_thread_count
option as well.RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR
to a lower value. RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR
is used for full GC trigger and the default is 2.0
.Do full GC when the number of old objects is more than R * N where R is this factor and N is the number of old objects just after last full GC.
2.0 * before old objects
. This improves the throughput but it grows the total memory usage. This setting is not good for the low resource environment e.g. a small container. For such cases, try RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR=0.9
or RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR=1.2
.multi workers
feature.