file - Fluentd

Search…
file
The out_file Output plugin writes events to files. By default, it creates files on a daily basis (around 00:10). This means that when you first import records using the plugin, no file is created immediately.
The file will be created when the timekey condition has been met. To change the output frequency, please modify the timekey value.
It is included in Fluentd's core.
Example Configuration
<match pattern>
  @type file
  path /var/log/fluent/myapp
  compress gzip
  <buffer>
    timekey 1d
    timekey_use_utc true
    timekey_wait 10m
  </buffer>
</match>
Please see the Configuration File article for the basic structure and syntax of the configuration file.
For <buffer>, refer to <buffer> Section.
Plugin Helpers
​formatter​
​inject​
​compat_parameters​
Parameters
​Common Parameters​
@type (required)
The value must be file.
path
type
default
version
string
required parameter
0.14.0
The path of the file. The actual path is path + time + ".log" by default. The path parameter supports placeholders, so you can embed time, tag and record fields in the path.
Here is an example:
path /path/to/${tag}/${key1}/file.%Y%m%d
<buffer tag,time,key1>
  # buffer parameters
</buffer>
See <buffer> Section for more detail.
The path parameter is used as <buffer>'s path in this plugin.
Initially, you may see a file which looks like /path/to/file.%Y%m%d/buffer.b5692238db04045286097f56f361028db.log. This is an intermediate buffer file (b5692238db04045286097f56f361028db identifies the buffer). Once the content of the buffer has been completely flushed, you will see the output file without the trailing identifier.
append
type
default
version
bool
false
0.14.0
Determines whether the flushed chunk is appended to an existing file or not. The default is not appended. By default, out_file flushes each chunk to a different path.
# append false
file.20140608.log_0
file.20140608.log_1
file.20140609.log_0
file.20140609.log_1
This makes parallel file processing easy. But if you want to disable this behavior, you can disable it by setting append true.
# append true
file.20140608.log
file.20140609.log
<format> Directive
The format of the file content. The default @type is out_file.
JSON example:
<format>
  @type json
</format>
See formatter article for more detail.
format
Deprecated parameter. Use <format> instead.
<inject> Section
Add event time and event tag to record.
See Inject Section Configurations for more details.
<buffer> Section
See Buffer Section Configurations for more details.
@type
type
default
version
string
file
0.14.9
Overwrites the default value in this plugin.
chunk_keys
type
default
version
array
time
0.14.9
Overwrites the default value in this plugin.
timekey
type
default
version
time
86400
0.14.9
Overwrites the default value in this plugin.
utc
Deprecated parameter. Use timekey_use_utc in <buffer> instead.
add_path_suffix
type
default
version
bool
true
0.14.9
Add path suffix or not. See also the path_suffix parameter.
path_suffix
type
default
version
string
".log"
0.14.9
The suffix for output result.
compress
Compresses flushed files using gzip. No compression is performed by default.
recompress
Performs compression again even if the buffer chunk is already compressed.
Default: false
symlink_path
Creates symlink to temporary buffered file when buffer_type is file. No symlink is created by default. This is useful for tailing file content to check logs.
This is disabled on Windows.
@log_level
The @log_level option allows the user to set different levels of logging for each plugin.
Supported log levels: fatal, error, warn, info, debug, trace
Please see the logging article for further details.
Common Output / Buffer parameters
For common output / buffer parameters, please check the following articles:
​Output Plugin Overview​
​Buffer Section Configuration​
FAQ
I can see files but placeholders are not replaced, why?
You see an intermediate buffer file, not the output result. The placeholders are replaced during flush buffers.
For example, if you have this setting:
path /path/to/file.${tag}.%Y%m%d
You see following buffer files first:
/path/to/file.${tag}.%Y%m%d/buffer.b5692238db04045286097f56f361028db.log
/path/to/file.${tag}.%Y%m%d/buffer.b5692238db04045286097f56f361028db.log.meta
After flushed, you see actual output result:
/path/to/file.test.20180405.log_0 # tag is 'test'
See the path parameter.
Can I use a placeholder in symlink_path?
Since Fluentd v1.4.0, you can use the placeholder syntax in symlink_path parameter.
For example, suppose you have the following configuration:
<source>
  @type dummy
  tag dummy1
</source>
​
<source>
  @type dummy
  tag dummy2
</source>
​
<match dummy*>
  @type file
  path /tmp/logs/${tag}
  symlink_path /tmp/logs/current-${tag}
  <buffer tag,time>
    @type file
  </buffer>
</match>
This produces the following directory layout:
$ tree /tmp/logs/
/tmp/logs/
├── ${tag}
│   ├── buffer.b57fb1dd96306dd0b308e094f7ec2228f.log
│   ├── buffer.b57fb1dd96306dd0b308e094f7ec2228f.log.meta
│   ├── buffer.b57fb1dd96339a870530991d4871cfe11.log
│   └── buffer.b57fb1dd96339a870530991d4871cfe11.log.meta
├── current-dummy1 -> /tmp/logs/${tag}/buffer.b57fb1dd96339a870530991d4871cfe11.log
└── current-dummy2 -> /tmp/logs/${tag}/buffer.b57fb1dd96306dd0b308e094f7ec2228f.log
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is an open-source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.
Output Plugins
forward
Last modified 7mo ago
Example Configuration

Plugin Helpers

Parameters

`@type` (required)

`path`

`append`

`<format>` Directive

`format`

`<inject>` Section

`<buffer>` Section

`@type`

`chunk_keys`

`timekey`

`utc`

`add_path_suffix`

`path_suffix`

`compress`

`recompress`

`symlink_path`

`@log_level`

Common Output / Buffer parameters

FAQ

I can see files but placeholders are not replaced, why?

Can I use a placeholder in `symlink_path`?
