Fluentd
1.0
1.0
  • Introduction
  • Overview
    • Life of a Fluentd event
    • Support
    • FAQ
    • Logo
    • fluent-package v5 vs td-agent v4
  • Installation
    • Before Installation
    • Install fluent-package
      • RPM Package (Red Hat Linux)
      • DEB Package (Debian/Ubuntu)
      • .dmg Package (macOS)
      • .msi Installer (Windows)
    • Install calyptia-fluentd
      • RPM Package (Red Hat Linux)
      • DEB Package (Debian/Ubuntu)
      • .dmg Package (macOS)
      • .msi Installer (Windows)
    • Install by Ruby Gem
    • Install from Source
    • Post Installation Guide
    • Obsolete Installation
      • Treasure Agent v4 (EOL) Installation
        • Install by RPM Package v4 (Red Hat Linux)
        • Install by DEB Package v4 (Debian/Ubuntu)
        • Install by .dmg Package v4 (macOS)
        • Install by .msi Installer v4 (Windows)
      • Treasure Agent v3 (EOL) Installation
        • Install by RPM Package v3 (Red Hat Linux)
        • Install by DEB Package v3 (Debian/Ubuntu)
        • Install by .dmg Package v3 (macOS)
        • Install by .msi Installer v3 (Windows)
  • Configuration
    • Config File Syntax
    • Config File Syntax (YAML)
    • Routing Examples
    • Config: Common Parameters
    • Config: Parse Section
    • Config: Buffer Section
    • Config: Format Section
    • Config: Extract Section
    • Config: Inject Section
    • Config: Transport Section
    • Config: Storage Section
    • Config: Service Discovery Section
  • Deployment
    • System Configuration
    • Logging
    • Signals
    • RPC
    • High Availability Config
    • Performance Tuning
    • Multi Process Workers
    • Failure Scenarios
    • Plugin Management
    • Trouble Shooting
    • Fluentd UI
    • Linux Capability
    • Command Line Option
    • Source Only Mode
    • Zero-downtime restart
  • Container Deployment
    • Docker Image
    • Docker Logging Driver
    • Docker Compose
    • Kubernetes
  • Monitoring Fluentd
    • Overview
    • Monitoring by Prometheus
    • Monitoring by REST API
  • Input Plugins
    • tail
    • forward
    • udp
    • tcp
    • unix
    • http
    • syslog
    • exec
    • sample
    • monitor_agent
    • windows_eventlog
  • Output Plugins
    • file
    • forward
    • http
    • exec
    • exec_filter
    • secondary_file
    • copy
    • relabel
    • roundrobin
    • stdout
    • null
    • s3
    • kafka
    • elasticsearch
    • opensearch
    • mongo
    • mongo_replset
    • rewrite_tag_filter
    • webhdfs
    • buffer
  • Filter Plugins
    • record_transformer
    • grep
    • parser
    • geoip
    • stdout
  • Parser Plugins
    • regexp
    • apache2
    • apache_error
    • nginx
    • syslog
    • ltsv
    • csv
    • tsv
    • json
    • msgpack
    • multiline
    • none
  • Formatter Plugins
    • out_file
    • json
    • ltsv
    • csv
    • msgpack
    • hash
    • single_value
    • stdout
    • tsv
  • Buffer Plugins
    • memory
    • file
    • file_single
  • Storage Plugins
    • local
  • Service Discovery Plugins
    • static
    • file
    • srv
  • Metrics Plugins
    • local
  • How-to Guides
    • Stream Analytics with Materialize
    • Send Apache Logs to S3
    • Send Apache Logs to Minio
    • Send Apache Logs to Mongodb
    • Send Syslog Data to Graylog
    • Send Syslog Data to InfluxDB
    • Send Syslog Data to Sematext
    • Data Analytics with Treasure Data
    • Data Collection with Hadoop (HDFS)
    • Simple Stream Processing with Fluentd
    • Stream Processing with Norikra
    • Stream Processing with Kinesis
    • Free Alternative To Splunk
    • Email Alerting like Splunk
    • How to Parse Syslog Messages
    • Cloud Data Logging with Raspberry Pi
  • Language Bindings
    • Java
    • Ruby
    • Python
    • Perl
    • PHP
    • Nodejs
    • Scala
  • Plugin Development
    • How to Write Input Plugin
    • How to Write Base Plugin
    • How to Write Buffer Plugin
    • How to Write Filter Plugin
    • How to Write Formatter Plugin
    • How to Write Output Plugin
    • How to Write Parser Plugin
    • How to Write Storage Plugin
    • How to Write Service Discovery Plugin
    • How to Write Tests for Plugin
    • Configuration Parameter Types
    • Upgrade Plugin from v0.12
  • Plugin Helper API
    • Plugin Helper: Child Process
    • Plugin Helper: Compat Parameters
    • Plugin Helper: Event Emitter
    • Plugin Helper: Event Loop
    • Plugin Helper: Extract
    • Plugin Helper: Formatter
    • Plugin Helper: Inject
    • Plugin Helper: Parser
    • Plugin Helper: Record Accessor
    • Plugin Helper: Server
    • Plugin Helper: Socket
    • Plugin Helper: Storage
    • Plugin Helper: Thread
    • Plugin Helper: Timer
    • Plugin Helper: Http Server
    • Plugin Helper: Service Discovery
  • Troubleshooting Guide
  • Appendix
    • Update from v0.12 to v1
    • td-agent v2 vs v3 vs v4
Powered by GitBook
On this page
  • Overview
  • Parameters
  • workers
  • restart_worker_interval
  • root_dir
  • log_level
  • suppress_repeated_stacktrace
  • emit_error_log_interval
  • ignore_repeated_log_interval
  • ignore_same_log_interval
  • suppress_config_dump
  • log_event_verbose
  • without_source
  • with_source_only
  • rpc_endpoint
  • enable_get_dump
  • process_name
  • enable_msgpack_time_support
  • file_permission
  • dir_permission
  • strict_config_value
  • disable_shared_socket
  • enable_jit (experimental)
  • <log> section
  • <source_only_buffer> section
  • Fluent::SystemConfig::Mixin Methods
  • Command-Line Options

Was this helpful?

  1. Deployment

System Configuration

This article describes Fluentd's system configurations for the <system> section and command-line options.

Overview

System Configuration is one way to set up system-wide configuration such as enabling RPC, multiple workers, etc.

Parameters

workers

type
default
version

integer

1

0.14.12

Specifies the number of workers.

restart_worker_interval

type
default
version

time

0

1.15.0

Specifies interval to restart workers that has stopped for some reason. By default, Fluentd immediately restarts stopped workers. You can use this option for stopping workers for a certain period of time.

root_dir

type
default
version

string

nil

0.14.11

Specifies the root directory.

log_level

type
default
version

enum

"info"

0.14.0

Specifies the log level e.g. trace, debug, info, warn, error, or fatal.

suppress_repeated_stacktrace

type
default
version

bool

nil

0.14.0

Suppresses the repeated stacktrace.

emit_error_log_interval

type
default
version

time

nil

0.14.0

Specifies the time value of the emitting of error log interval.

ignore_repeated_log_interval

type
default
version

time

nil

1.10.2

Specify interval to ignore repeated log/stacktrace messages.

ignore_same_log_interval

type
default
version

time

nil

1.11.3

Specify interval to ignore same log/stacktrace messages.

suppress_config_dump

type
default
version

bool

nil

0.14.0

Suppresses the configuration dump.

log_event_verbose

type
default
version

bool

nil

0.14.12

Logs event verbosely.

without_source

type
default
version

bool

nil

0.14.0

Invokes fluentd without input plugins.

with_source_only

type
default
version

bool

nil

1.18.0

Not supported on Windows.

Invokes a fluentd only with input plugins.

rpc_endpoint

type
default
version

string

nil

0.14.0

enable_get_dump

type
default
version

bool

nil

0.14.0

Enables to get dump.

process_name

type
default
version

string

nil

0.14.0

Specifies the process name.

enable_msgpack_time_support

type
default
version

bool

false

1.9.0

Allows Time object in buffer's MessagePack serde. This is useful when the log contains multiple time fields.

file_permission

type
default
version

string

nil

0.14.0

Specifies the file permission in the octal format.

dir_permission

type
default
version

string

nil

0.14.0

Specifies the directory permission in the octal format.

strict_config_value

type
default
version

bool

nil

1.8.0

Parses config values strictly. Invalid numerical or boolean values are not allowed. Fluentd raises configuration error instead of replacing them with 0 or true implicitly.

disable_shared_socket

type
default
version

bool

nil

1.12.1

Force disable the shared socket which is for listening a same port across multiple worker processes. When the shared socket is enabled (it's the default behavior), a socket is always created to enable workers to communicate with the supervisor. On Windows, it consumes a dynamic (a.k.a ephemeral) TCP port. If you don't prefer it, set this option as true. When it's disabled, you may not use plugins that listen a port such as in_forward, in_http and in_syslog.

enable_jit (experimental)

type
default
version

bool

false

1.15.2

Enable JIT for worker processes. Internally, this configuration enables Ruby's --jit command-line option.

<log> section

path

type
default
version

string

nil

1.18.0

Specifies the log file path.

format

type
default
version

enum

text

0.14.20

Specifies the logging format e.g. text or json.

time_format

type
default
version

string

%Y-%m-%d %H:%M:%S %z

0.14.20

Specifies time format.

rotate_age

type
default
version

enum or integer

7

1.13.0

Specifies daily, weekly, monthly or integer which indicates age of log rotation.

By default, Fluentd does not rotate log files. You need to specify rotate_age or rotate_size options explicitly to enable log rotation.

NOTE: When enabling log rotation on Windows, log files are separated into log-supervisor-0.log, log-0.log, ..., log-N.log where N is generation - 1 due to the system limitation. Windows does not permit delete and rename files simultaneously owned by another process.

rotate_size

type
default
version

size

1048576

1.13.0

Specifies log file size limitation.

By default, Fluentd does not rotate log files. You need to specify rotate_age or rotate_size options explicitly to enable log rotation.

NOTE: When enabling log rotation on Windows, log files are separated into log-supervisor-0.log, log-0.log, ..., log-N.log where N is generation - 1 due to the system limitation. Windows does not permit delete and rename files simultaneously owned by another process.

enable_input_metrics

type
default
version

bool

nil

1.14.0

Specifies whether measuring input metrics should be enabled or not.

enable_size_metrics

type
default
version

bool

nil

1.14.0

Specifies whether measuring record size metrics should be enabled or not. It can be useful for calculating flow rate on Fluentd instances.

<source_only_buffer> section

flush_thread_count

type
default
version

integer

1

1.18.0

overflow_action

type
default
version

enum

drop_oldest_chunk

1.18.0

path

type
default
version

string

nil (use unique path for the instance)

1.18.0

flush_interval

type
default
version

integer

10

1.18.0

chunk_limit_size

type
default
version

size

256MB

1.18.0

total_limit_size

type
default
version

size

64GB

1.18.0

compress

type
default
version

enum

text

1.18.0

Fluent::SystemConfig::Mixin Methods

.system_config

Returns Fluent::SystemConfig instance.

.system_config_override(options = {})

Overwrites the system configuration.

This is for internal use and plugin testing.

Command-Line Options

  • -s, --setup [DIR=/etc/fluent]: Installs sample configuration file to the directory.

  • -c, --config PATH: Specifies the configuration file path. (Default: /etc/fluent/fluent.conf)

  • --dry-run: Verifies the fluentd setup without launching.

  • --show-plugin-config=PLUGIN (obsolete): Use fluent-plugin-config-format command instead.

  • -p, --plugin DIR: Specifies the plugins' directory.

  • -I PATH: Specifies the library path.

  • -r NAME: Loads the library.

  • -d, --daemon PIDFILE: Daemonizes the fluentd process.

  • --under-supervisor: Runs the fluent worker under supervisor (this option is NOT for users).

  • --no-supervisor: Runs the fluent worker without supervisor.

  • --workers NUM: Specifies the number of workers under supervisor.

  • --user USER: Changes user.

  • --group GROUP: Changes group.

  • --umask UMASK: Specifies umask to use, in octal form. Defaults to 0.

  • -o, --log PATH: Specifies the log file path.

  • --log-rotate-age AGE: Specifies the generations to keep rotated log files.

  • --log-rotate-size BYTES: Sets the byte size to rotate log files.

  • --log-event-verbose: Enables the events logging during startup/shutdown.

  • -i CONFIG_STRING, --inline-config CONFIG_STRING: Specifies the inlined configuration which is appended to the config file on-the-fly.

  • --emit-error-log-interval SECONDS: Suppresses the interval (seconds) to emit error logs.

  • --suppress-repeated-stacktrace [VALUE]: Suppress the repeated stacktrace.

  • --without-source: Invokes fluentd without input plugins.

  • --use-v1-config: Uses v1 configuration format (default).

  • --use-v0-config: Uses v0 configuration format.

  • --strict-config-value: Parses the configuration values strictly. Invalid numerical or boolean values are rejected. Fluentd raises configuration error exceptions instead of replacing them with 0 or true implicitly.

  • -v, --verbose: Increases the verbosity level (-v: debug, -vv: trace).

  • -q, --quiet: Decreases the verbosity level (-q: warn, -qq: error).

  • --suppress-config-dump: Suppresses the configuration dumping on start.

  • -g, --gemfile GEMFILE: Specifies the Gemfile path.

  • -G, --gem-path GEM_INSTALL_PATH: Specifies the Gemfile install path. (Default: $(dirname $gemfile)/vendor/bundle)

PreviousDeploymentNextLogging

Last updated 4 months ago

Was this helpful?

See also .

See also .

See for details.

See also .

Specifies an RPC endpoint. Refer to the article for more detail.

Please see also .

Please see also .

Please see also .

The temporary buffer setting for .

See for details.

See for details.

Set this option when recovering buffers ().

See for details.

See for details.

See for details.

See for details.

See for details.

For code example, please refer to 's .system_config description.

For code example, please refer to 's .system_config_override description.

--with-source-only (Not supported on Windows): Invokes a fluentd only with input plugins. See for details.

If this article is incorrect or outdated, or omits critical information, please . is an open-source project under . All components are available under the Apache 2 License.

Source Only Mode
RPC
Source Only Mode
api-plugin-base
api-plugin-base
Source Only Mode
let us know
Fluentd
Cloud Native Computing Foundation (CNCF)
source_only_buffer section
Source Only Mode - Recovery
Logging article
Logging article
Output to Log File
Log Rotation Setting
Log Rotation Setting
Buffer Plugins - file - path
Buffer Section Configurations - Buffering Parameters
Buffer Section Configurations - Buffering Parameters
Buffer Section Configurations - Buffering Parameters
Output Plugins - flush_thread_count
Output Plugins - overflow_action
Output Plugins - flush_interval