The out_secondary_file Output plugin writes chunks to files. This plugin is similar to out_file but this is for <secondary> use-case.

NOTE: Do not use this plugin for the primary plugin.

out_secondary_file is included in Fluentd's core.

Example Configuration

<match pattern>
  @type forward
  # ...
  <secondary>
    @type secondary_file
    directory /var/log/fluentd/error
  </secondary>
</match>

With this configuration, failed buffer chunks are saved into /var/log/fluentd/error/dump.bin.N, N is 0-origin incremental number.

Please see the Configuration File article for the basic structure and syntax of the configuration file.

Plugin Helpers

No helpers.

Parameters

​Common Parameters​

@type (required)

The value must be secondary_file.

directory

The directory path of the output file. Received buffer chunks are saved in this directory.

<secondary>
  @type secondary_file
  directory /var/log/fluentd/error
</secondary>

basename

The basename of the output file. You can use ${chunk_id} placeholder to identify the original chunk.

<secondary>
  @type secondary_file
  directory /var/log/fluentd/error
  basename dump.${chunk_id}
</secondary>

The output path would be:

# 59c278456e74a22dc594b06a7d4247c4 is chunk id
/var/log/fluentd/error/dump.59c278456e74a22dc594b06a7d4247c4.0

append

Determines that the received chunk is appended to an existing file or not. By default, it is not appended and each chunk is flushed to a different path:

# append false
dump.bin.0
dump.bin.1
dump.bin.2
...
dump.bin.N

This makes parallel file processing easy. But if you want to disable this behavior, you can disable it by setting append true:

# append true
dump.bin

How to resend secondary file

The secondary file can be resend by fluent-cat command.

Here is the example to resend dump.bin.0 to 127.0.0.1:24224 using fluent-cat.

$ cat /path/to/dump.bin.0 | fluent-cat --format msgpack  --host 127.0.0.1 --port 24224 TAG

See fluent-cat command line option about details.

If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is an open-source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.