secondary_file

Search…

secondary_file

The out_secondary_file Output plugin writes chunks to files. This plugin is similar to out_file but this is for <secondary> use-case.
NOTE: Do not use this plugin for the primary plugin.
out_secondary_file is included in Fluentd's core.
Example Configuration
<match pattern>
  @type forward
  # ...
  <secondary>
    @type secondary_file
    directory /var/log/fluentd/error
  </secondary>
</match>
With this configuration, failed buffer chunks are saved into /var/log/fluentd/error/dump.bin.N, N is 0-origin incremental number.
Please see the Configuration File article for the basic structure and syntax of the configuration file.
Plugin Helpers
No helpers.
Parameters
​Common Parameters​
@type (required)
The value must be secondary_file.
directory
type
default
version
string
required parameter
1.0.0
The directory path of the output file. Received buffer chunks are saved in this directory.
<secondary>
  @type secondary_file
  directory /var/log/fluentd/error
</secondary>
basename
type
default
version
string
dump.bin
1.0.0
The basename of the output file. You can use ${chunk_id} placeholder to identify the original chunk.
<secondary>
  @type secondary_file
  directory /var/log/fluentd/error
  basename dump.${chunk_id}
</secondary>
The output path would be:
# 59c278456e74a22dc594b06a7d4247c4 is chunk id
/var/log/fluentd/error/dump.59c278456e74a22dc594b06a7d4247c4.0
append
type
default
version
bool
false
1.0.0
Determines that the received chunk is appended to an existing file or not. By default, it is not appended and each chunk is flushed to a different path:
# append false
dump.bin.0
dump.bin.1
dump.bin.2
...
dump.bin.N
This makes parallel file processing easy. But if you want to disable this behavior, you can disable it by setting append true:
# append true
dump.bin
How to resend secondary file
The secondary file can be resend by fluent-cat command.
Here is the example to resend dump.bin.0 to 127.0.0.1:24224 using fluent-cat.
$ cat /path/to/dump.bin.0 | fluent-cat --format msgpack  --host 127.0.0.1 --port 24224 TAG
See fluent-cat command line option about details.
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is an open-source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.

exec_filter

copy

Last modified 1yr ago

Copy link

On this page

Example Configuration

Plugin Helpers

Parameters

@type (required)

Example Configuration

Plugin Helpers

Parameters

`@type` (required)

`directory`

`basename`

`append`

How to resend secondary file