file
The
file
buffer plugin provides a persistent buffer implementation. It uses files to store buffer chunks on disk.<match pattern>
buffer_type file
buffer_path /var/log/fluent/myapp.*.buffer
</match>
The value must be
file
.The path where buffer chunks are stored. The '*' is replaced with random characters. This parameter is require
This parameter must be unique to avoid race condition problem. For example, you can't use fixed buffer_path parameter in fluent-plugin-forest.
${tag}
or similar placeholder is needed. Of course, this parameter must also be unique between fluentd instances.In addition,
buffer_path
should not be an other buffer_path
prefix. For example, the following conf doesn't work well. /var/log/fluent/foo
resumes /var/log/fluent/foo.bar
's buffer files during start phase and it causes No such file or directory
in /var/log/fluent/foo.bar
side.<match pattern1>
buffer_path /var/log/fluent/foo
</match>
<match pattern2>
buffer_path /var/log/fluent/foo.bar
</match>
Here is the correct version to avoid prefix problem.
<match pattern1>
buffer_path /var/log/fluent/foo.baz
</match>
<match pattern2>
buffer_path /var/log/fluent/foo.bar
</match>
The size of each buffer chunk. The default is 8m. The suffixes "k" (KB), "m" (MB), and "g" (GB) can be used. Please see the Buffer Plugin Overview article for the basic buffer structure.
The default value for Time Sliced Plugin is overwritten as 256m.
The length limit of the chunk queue. Please see the Buffer Plugin Overview article for the basic buffer structure. The default limit is 256 chunks.
The interval between data flushes. The suffixes "s" (seconds), "m" (minutes), and "h" (hours) can be used
If true, queued chunks are flushed at shutdown process. The default is
false
.The interval between retries. The suffixes "s" (seconds), "m" (minutes), and "h" (hours) can be used.
Caution:
file
buffer implementation depends on the characteristics of local file system. Don't use file
buffer on remote file system, e.g. NFS, GlusterFS, HDFS and etc. We observed major data loss by using remote file system.If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.
Last modified 2yr ago