Fluentd
1.0
1.0
  • Introduction
  • Overview
    • Life of a Fluentd event
    • Support
    • FAQ
    • Logo
    • fluent-package v5 vs td-agent v4
  • Installation
    • Before Installation
    • Install fluent-package
      • RPM Package (Red Hat Linux)
      • DEB Package (Debian/Ubuntu)
      • .dmg Package (macOS)
      • .msi Installer (Windows)
    • Install calyptia-fluentd
      • RPM Package (Red Hat Linux)
      • DEB Package (Debian/Ubuntu)
      • .dmg Package (macOS)
      • .msi Installer (Windows)
    • Install by Ruby Gem
    • Install from Source
    • Post Installation Guide
    • Obsolete Installation
      • Treasure Agent v4 (EOL) Installation
        • Install by RPM Package v4 (Red Hat Linux)
        • Install by DEB Package v4 (Debian/Ubuntu)
        • Install by .dmg Package v4 (macOS)
        • Install by .msi Installer v4 (Windows)
      • Treasure Agent v3 (EOL) Installation
        • Install by RPM Package v3 (Red Hat Linux)
        • Install by DEB Package v3 (Debian/Ubuntu)
        • Install by .dmg Package v3 (macOS)
        • Install by .msi Installer v3 (Windows)
  • Configuration
    • Config File Syntax
    • Config File Syntax (YAML)
    • Routing Examples
    • Config: Common Parameters
    • Config: Parse Section
    • Config: Buffer Section
    • Config: Format Section
    • Config: Extract Section
    • Config: Inject Section
    • Config: Transport Section
    • Config: Storage Section
    • Config: Service Discovery Section
  • Deployment
    • System Configuration
    • Logging
    • Signals
    • RPC
    • High Availability Config
    • Performance Tuning
    • Multi Process Workers
    • Failure Scenarios
    • Plugin Management
    • Trouble Shooting
    • Fluentd UI
    • Linux Capability
    • Command Line Option
    • Source Only Mode
    • Zero-downtime restart
  • Container Deployment
    • Docker Image
    • Docker Logging Driver
    • Docker Compose
    • Kubernetes
  • Monitoring Fluentd
    • Overview
    • Monitoring by Prometheus
    • Monitoring by REST API
  • Input Plugins
    • tail
    • forward
    • udp
    • tcp
    • unix
    • http
    • syslog
    • exec
    • sample
    • monitor_agent
    • windows_eventlog
  • Output Plugins
    • file
    • forward
    • http
    • exec
    • exec_filter
    • secondary_file
    • copy
    • relabel
    • roundrobin
    • stdout
    • null
    • s3
    • kafka
    • elasticsearch
    • opensearch
    • mongo
    • mongo_replset
    • rewrite_tag_filter
    • webhdfs
    • buffer
  • Filter Plugins
    • record_transformer
    • grep
    • parser
    • geoip
    • stdout
  • Parser Plugins
    • regexp
    • apache2
    • apache_error
    • nginx
    • syslog
    • ltsv
    • csv
    • tsv
    • json
    • msgpack
    • multiline
    • none
  • Formatter Plugins
    • out_file
    • json
    • ltsv
    • csv
    • msgpack
    • hash
    • single_value
    • stdout
    • tsv
  • Buffer Plugins
    • memory
    • file
    • file_single
  • Storage Plugins
    • local
  • Service Discovery Plugins
    • static
    • file
    • srv
  • Metrics Plugins
    • local
  • How-to Guides
    • Stream Analytics with Materialize
    • Send Apache Logs to S3
    • Send Apache Logs to Minio
    • Send Apache Logs to Mongodb
    • Send Syslog Data to Graylog
    • Send Syslog Data to InfluxDB
    • Send Syslog Data to Sematext
    • Data Analytics with Treasure Data
    • Data Collection with Hadoop (HDFS)
    • Simple Stream Processing with Fluentd
    • Stream Processing with Norikra
    • Stream Processing with Kinesis
    • Free Alternative To Splunk
    • Email Alerting like Splunk
    • How to Parse Syslog Messages
    • Cloud Data Logging with Raspberry Pi
  • Language Bindings
    • Java
    • Ruby
    • Python
    • Perl
    • PHP
    • Nodejs
    • Scala
  • Plugin Development
    • How to Write Input Plugin
    • How to Write Base Plugin
    • How to Write Buffer Plugin
    • How to Write Filter Plugin
    • How to Write Formatter Plugin
    • How to Write Output Plugin
    • How to Write Parser Plugin
    • How to Write Storage Plugin
    • How to Write Service Discovery Plugin
    • How to Write Tests for Plugin
    • Configuration Parameter Types
    • Upgrade Plugin from v0.12
  • Plugin Helper API
    • Plugin Helper: Child Process
    • Plugin Helper: Compat Parameters
    • Plugin Helper: Event Emitter
    • Plugin Helper: Event Loop
    • Plugin Helper: Extract
    • Plugin Helper: Formatter
    • Plugin Helper: Inject
    • Plugin Helper: Parser
    • Plugin Helper: Record Accessor
    • Plugin Helper: Server
    • Plugin Helper: Socket
    • Plugin Helper: Storage
    • Plugin Helper: Thread
    • Plugin Helper: Timer
    • Plugin Helper: Http Server
    • Plugin Helper: Service Discovery
  • Troubleshooting Guide
  • Appendix
    • Update from v0.12 to v1
    • td-agent v2 vs v3 vs v4
Powered by GitBook
On this page
  • Example Configuration
  • Plugin Helpers
  • Parameters
  • @type (required)
  • path
  • append
  • <format> Directive
  • format
  • <inject> Section
  • <buffer> Section
  • utc
  • add_path_suffix
  • path_suffix
  • compress
  • recompress
  • symlink_path
  • Common Output / Buffer parameters
  • FAQ
  • I can see files but placeholders are not replaced, why?
  • Can I use a placeholder in symlink_path?

Was this helpful?

  1. Output Plugins

file

PreviousOutput PluginsNextforward

Last updated 3 years ago

Was this helpful?

The out_file Output plugin writes events to files. By default, it creates files on a daily basis (around 00:10). This means that when you first import records using the plugin, no file is created immediately.

The file will be created when the timekey condition has been met. To change the output frequency, please modify the timekey value.

It is included in Fluentd's core.

Example Configuration

<match pattern>
  @type file
  path /var/log/fluent/myapp
  compress gzip
  <buffer>
    timekey 1d
    timekey_use_utc true
    timekey_wait 10m
  </buffer>
</match>

Plugin Helpers

Parameters

@type (required)

The value must be file.

path

type
default
version

string

required parameter

0.14.0

The path of the file. The actual path is path + time + ".log" by default. The path parameter supports placeholders, so you can embed time, tag and record fields in the path.

Here is an example:

path /path/to/${tag}/${key1}/file.%Y%m%d
<buffer tag,time,key1>
  # buffer parameters
</buffer>

The path parameter is used as <buffer>'s path in this plugin.

append

type
default
version

bool

false

0.14.0

Determines whether the flushed chunk is appended to an existing file or not. The default is not appended. By default, out_file flushes each chunk to a different path.

# append false
file.20140608.log_0
file.20140608.log_1
file.20140609.log_0
file.20140609.log_1

This makes parallel file processing easy. But if you want to disable this behavior, you can disable it by setting append true.

# append true
file.20140608.log
file.20140609.log

<format> Directive

The format of the file content. The default @type is out_file.

JSON example:

<format>
  @type json
</format>

format

Deprecated parameter. Use <format> instead.

<inject> Section

Add event time and event tag to record.

<buffer> Section

@type

type
default
version

string

file

0.14.9

Overwrites the default value in this plugin.

chunk_keys

type
default
version

array

time

0.14.9

Overwrites the default value in this plugin.

timekey

type
default
version

time

86400

0.14.9

Overwrites the default value in this plugin.

utc

Deprecated parameter. Use timekey_use_utc in <buffer> instead.

add_path_suffix

type
default
version

bool

true

0.14.9

Add path suffix or not. See also the path_suffix parameter.

path_suffix

type
default
version

string

".log"

0.14.9

The suffix for output result.

compress

Compresses flushed files using gzip. No compression is performed by default.

recompress

Performs compression again even if the buffer chunk is already compressed.

Default: false

symlink_path

Creates symlink to temporary buffered file when buffer_type is file. No symlink is created by default. This is useful for tailing file content to check logs.

This is disabled on Windows.

@log_level

The @log_level option allows the user to set different levels of logging for each plugin.

Supported log levels: fatal, error, warn, info, debug, trace

Common Output / Buffer parameters

For common output / buffer parameters, please check the following articles:

FAQ

I can see files but placeholders are not replaced, why?

You see an intermediate buffer file, not the output result. The placeholders are replaced during flush buffers.

For example, if you have this setting:

path /path/to/file.${tag}.%Y%m%d

You see following buffer files first:

/path/to/file.${tag}.%Y%m%d/buffer.b5692238db04045286097f56f361028db.log
/path/to/file.${tag}.%Y%m%d/buffer.b5692238db04045286097f56f361028db.log.meta

After flushed, you see actual output result:

/path/to/file.test.20180405.log_0 # tag is 'test'

See the path parameter.

Can I use a placeholder in symlink_path?

Since Fluentd v1.4.0, you can use the placeholder syntax in symlink_path parameter.

For example, suppose you have the following configuration:

<source>
  @type dummy
  tag dummy1
</source>

<source>
  @type dummy
  tag dummy2
</source>

<match dummy*>
  @type file
  path /tmp/logs/${tag}
  symlink_path /tmp/logs/current-${tag}
  <buffer tag,time>
    @type file
  </buffer>
</match>

This produces the following directory layout:

$ tree /tmp/logs/
/tmp/logs/
├── ${tag}
│   ├── buffer.b57fb1dd96306dd0b308e094f7ec2228f.log
│   ├── buffer.b57fb1dd96306dd0b308e094f7ec2228f.log.meta
│   ├── buffer.b57fb1dd96339a870530991d4871cfe11.log
│   └── buffer.b57fb1dd96339a870530991d4871cfe11.log.meta
├── current-dummy1 -> /tmp/logs/${tag}/buffer.b57fb1dd96339a870530991d4871cfe11.log
└── current-dummy2 -> /tmp/logs/${tag}/buffer.b57fb1dd96306dd0b308e094f7ec2228f.log

Please see the article for the basic structure and syntax of the configuration file.

For <buffer>, refer to .

See for more detail.

Initially, you may see a file which looks like /path/to/file.%Y%m%d/buffer.b5692238db04045286097f56f361028db.log. This is an intermediate buffer file (b5692238db04045286097f56f361028db identifies the buffer). Once the content of the buffer has been completely , you will see the output file without the trailing identifier.

See article for more detail.

See for more details.

See for more details.

Please see the article for further details.

If this article is incorrect or outdated, or omits critical information, please . is an open-source project under . All components are available under the Apache 2 License.

Configuration File
formatter
inject
compat_parameters
Common Parameters
flushed
formatter
Inject Section Configurations
Buffer Section Configurations
logging
Output Plugin Overview
Buffer Section Configuration
let us know
Fluentd
Cloud Native Computing Foundation (CNCF)
<buffer> Section
<buffer> Section