file
Last updated
Was this helpful?
Last updated
Was this helpful?
The file
buffer plugin provides a persistent buffer implementation. It uses files to store buffer chunks on disk.
path
string
nil
0.9.0
The directory path where buffer chunks are stored. Don't share this directory path with other buffers. Be sure to specify a unique path for each buffer.
This config outputs the buffer chunk files as follows. The file name is buffer.b{chunk_id}{path_suffix}
.
This config outputs the buffer chunk files as follows. The directory worker{worker_id}
is automatically created.
This config outputs the buffer chunk files as follows. The directory {root_dir}/worker{worker_id}/{@id}/buffer
is used for the path. In this case, the worker{worker_id}
directory is created even for a single worker.
Please make sure that you have enough space in the path directory. Running out of disk space is a problem frequently reported by users.
path_suffix
string
.log
1.6.3
Changes the suffix of the buffer file.
You can customize the prefix of filename (buffer
by default) by adding .*
to the end of the path
parameter.
This config outputs the buffer chunk files as follows. The prefix buffer
is changed to custom
.
You can also customize the entire filename by adding .*.
to the path
parameter.
This config outputs the buffer chunk files as follows. In this case, path_suffix
parameter is not used.
Caution: file
buffer implementation depends on the characteristics of the local file system. Don't use file
buffer on remote file systems e.g. NFS, GlusterFS, HDFS, etc. We observed major data loss by using the remote file system.
With , a directory is automatically created for each worker. So there is no need to specify a unique path for each worker.
If you specify root_dir
in and of the plugin, then you can omit this parameter.
This parameter is useful when .log
is not fit for your environment. See also .
If this article is incorrect or outdated, or omits critical information, please . is an open-source project under . All components are available under the Apache 2 License.