Fluentd
1.0
1.0
  • Introduction
  • Overview
    • Life of a Fluentd event
    • Support
    • FAQ
    • Logo
    • fluent-package v5 vs td-agent v4
  • Installation
    • Before Installation
    • Install fluent-package
      • RPM Package (Red Hat Linux)
      • DEB Package (Debian/Ubuntu)
      • .dmg Package (macOS)
      • .msi Installer (Windows)
    • Install calyptia-fluentd
      • RPM Package (Red Hat Linux)
      • DEB Package (Debian/Ubuntu)
      • .dmg Package (macOS)
      • .msi Installer (Windows)
    • Install by Ruby Gem
    • Install from Source
    • Post Installation Guide
    • Obsolete Installation
      • Treasure Agent v4 (EOL) Installation
        • Install by RPM Package v4 (Red Hat Linux)
        • Install by DEB Package v4 (Debian/Ubuntu)
        • Install by .dmg Package v4 (macOS)
        • Install by .msi Installer v4 (Windows)
      • Treasure Agent v3 (EOL) Installation
        • Install by RPM Package v3 (Red Hat Linux)
        • Install by DEB Package v3 (Debian/Ubuntu)
        • Install by .dmg Package v3 (macOS)
        • Install by .msi Installer v3 (Windows)
  • Configuration
    • Config File Syntax
    • Config File Syntax (YAML)
    • Routing Examples
    • Config: Common Parameters
    • Config: Parse Section
    • Config: Buffer Section
    • Config: Format Section
    • Config: Extract Section
    • Config: Inject Section
    • Config: Transport Section
    • Config: Storage Section
    • Config: Service Discovery Section
  • Deployment
    • System Configuration
    • Logging
    • Signals
    • RPC
    • High Availability Config
    • Performance Tuning
    • Multi Process Workers
    • Failure Scenarios
    • Plugin Management
    • Trouble Shooting
    • Fluentd UI
    • Linux Capability
    • Command Line Option
    • Source Only Mode
    • Zero-downtime restart
  • Container Deployment
    • Docker Image
    • Docker Logging Driver
    • Docker Compose
    • Kubernetes
  • Monitoring Fluentd
    • Overview
    • Monitoring by Prometheus
    • Monitoring by REST API
  • Input Plugins
    • tail
    • forward
    • udp
    • tcp
    • unix
    • http
    • syslog
    • exec
    • sample
    • monitor_agent
    • windows_eventlog
  • Output Plugins
    • file
    • forward
    • http
    • exec
    • exec_filter
    • secondary_file
    • copy
    • relabel
    • roundrobin
    • stdout
    • null
    • s3
    • kafka
    • elasticsearch
    • opensearch
    • mongo
    • mongo_replset
    • rewrite_tag_filter
    • webhdfs
    • buffer
  • Filter Plugins
    • record_transformer
    • grep
    • parser
    • geoip
    • stdout
  • Parser Plugins
    • regexp
    • apache2
    • apache_error
    • nginx
    • syslog
    • ltsv
    • csv
    • tsv
    • json
    • msgpack
    • multiline
    • none
  • Formatter Plugins
    • out_file
    • json
    • ltsv
    • csv
    • msgpack
    • hash
    • single_value
    • stdout
    • tsv
  • Buffer Plugins
    • memory
    • file
    • file_single
  • Storage Plugins
    • local
  • Service Discovery Plugins
    • static
    • file
    • srv
  • Metrics Plugins
    • local
  • How-to Guides
    • Stream Analytics with Materialize
    • Send Apache Logs to S3
    • Send Apache Logs to Minio
    • Send Apache Logs to Mongodb
    • Send Syslog Data to Graylog
    • Send Syslog Data to InfluxDB
    • Send Syslog Data to Sematext
    • Data Analytics with Treasure Data
    • Data Collection with Hadoop (HDFS)
    • Simple Stream Processing with Fluentd
    • Stream Processing with Norikra
    • Stream Processing with Kinesis
    • Free Alternative To Splunk
    • Email Alerting like Splunk
    • How to Parse Syslog Messages
    • Cloud Data Logging with Raspberry Pi
  • Language Bindings
    • Java
    • Ruby
    • Python
    • Perl
    • PHP
    • Nodejs
    • Scala
  • Plugin Development
    • How to Write Input Plugin
    • How to Write Base Plugin
    • How to Write Buffer Plugin
    • How to Write Filter Plugin
    • How to Write Formatter Plugin
    • How to Write Output Plugin
    • How to Write Parser Plugin
    • How to Write Storage Plugin
    • How to Write Service Discovery Plugin
    • How to Write Tests for Plugin
    • Configuration Parameter Types
    • Upgrade Plugin from v0.12
  • Plugin Helper API
    • Plugin Helper: Child Process
    • Plugin Helper: Compat Parameters
    • Plugin Helper: Event Emitter
    • Plugin Helper: Event Loop
    • Plugin Helper: Extract
    • Plugin Helper: Formatter
    • Plugin Helper: Inject
    • Plugin Helper: Parser
    • Plugin Helper: Record Accessor
    • Plugin Helper: Server
    • Plugin Helper: Socket
    • Plugin Helper: Storage
    • Plugin Helper: Thread
    • Plugin Helper: Timer
    • Plugin Helper: Http Server
    • Plugin Helper: Service Discovery
  • Troubleshooting Guide
  • Appendix
    • Update from v0.12 to v1
    • td-agent v2 vs v3 vs v4
Powered by GitBook
On this page
  • Types of Configuration Parameters
  • Common Options
  • Data Types
  • :string
  • :regexp
  • :integer
  • :float
  • :size
  • :time
  • :bool
  • :enum
  • :array
  • :hash

Was this helpful?

  1. Plugin Development

Configuration Parameter Types

Types of Configuration Parameters

Common Options

  • default: Specifies the default value for a parameter. If omitted, the

    parameter is required. On startup, Fluentd uses the default value instead if

    the parameter is not configured.

# Required parameter: The configuration must have this parameter like 'param1 10'.
config_param :param1, :integer

# Optional parameter: If the configuration doesn't have 'param2', 100 is used.
config_param :param2, :integer, default: 100
  • secret: If true, the parameter will be masked when Fluentd dumps its

    configuration on the standard output on startup.

config_param :secret_param, :string, secret: true
  • deprecated: Specifies the deprecation warning message. If users use this

    parameter in the configuration, they will see the deprecation warning on

    startup.

config_param :old_param, :string, deprecated: "Use new_param instead"
  • obsoleted: Specifies the obsolete error message. If users use this parameter

    in the configuration, Fluentd raises Fluent::ConfigError and stops.

config_param :dead_param, :string, obsoleted: "This parameter doesn't work anymore"
  • alias: Alias for this parameter as a symbol.

  • skip_accessor: If true, skip adding accessor to the plugin. For internal

    use only!

Data Types

:string

Defines a string parameter.

Code Example:

config_param :name, :string, default: "John Doe", alias: :full_name
config_param :password, :string, secret: true

def configure(conf)
  super

  log.info(name: @name, password: @password)
end

Configuration Example:

name John Titor
password very-secret-password

:regexp

Defines a regexp parameter. Since v1.2.0.

Code Example:

config_param :pattern, :regexp, default: /^key_/

def configure(conf)
  super

  log.info(pattern: @pattern)
end

def filter(tag, time, record)
  new_record = record.select do |k, v|
    @pattern.match(k)
  end
  new_record
end

Configuration Example:

pattern /^name_/
pattern ^name_    # Also support pattern without slashes

:integer

Defines an integer parameter.

Code Example:

config_param :num_children, :integer, default: 1

def start
  super
  # ...
  @num_children.times do |n|
    # do something...
  end
end

Configuration Example:

num_children 10

:float

Defines a float parameter.

Code Example:

helpers :timer
config_param :interval, :float, default: 0.5

def start
  super
  # ...
  timer_execute(:in_example, @interval) do
    # do something periodically
  end
end

Configuration Example:

interval 1.5

:size

Defines a size parameter in bytes.

Available suffixes: { k, m, g, t } (ignore case)

Code Example:

config_param :limit, :size

def do_something
  raise "overflow!" if @limit < current
  # ...
end

Configuration Example:

limit 10  # 10 byte
limit 10k # 10240 byte
limit 10m # 10485760 byte
limit 10g # 10737418240 byte
limit 10t # 10995116277760 byte

Configuration Example:

limit 10m

:time

Defines the length of the time parameter.

Available suffixes: { s, m, h, d } (lower case only).

If omitted, to_f is applied to the value which converts it to seconds.

Code Example:

config_param :interval, :time

def start
  timer_execute(:in_example, @interval) do
    # do something periodically
  end
end

Configuration Example:

interval 0.5 # 0.5 seconds
interval 1s  # 1 second
interval 1m  # 1 minute = 60 seconds
interval 1h  # 1 hour = 3600 seconds
interval 1d  # 1 day = 86400 seconds

Configuration Example:

interval 10m

:bool

Defines a Boolean parameter.

Code Example:

config_param :deep_copy, :bool, default: false

def copy(object)
  if @deep_copy
    # deep copy
  else
    # shallow copy
  end
end

Configuration Example:

deep_copy true

:enum

Defines an enumerated parameter.

Users can choose a value from the list. If a non-listed value is chosen, an error occurs on startup.

  • Available options

    • list: List of available values.

Code Example:

config_param :protocol_type, :enum, list: [:udp, :tcp], default: :udp

def send
  case @protocol_type
  when :udp
    send_udp
  when :tcp
    send_tcp
  end
end

Configuration Example:

protocol_type tcp

:array

Defines an array parameter.

Users can set an array value for a parameter.

  • Available options

    • value_type: Defines the type of the value.

      Available types: { :string, :integer, :float, :size, :bool, :time }

Code Example:

config_param :users, :array, default: [], value_type: :string

def available_user?(user)
  @users.include?(user)
end

Configuration Example:

users user1, user2, user3
users ["user1", "user2", "user3"] # written in JSON

This configuration will be converted to:

["user1", "user2", "user3"]

:hash

Defines a hash parameter.

  • Available options

    • symbolize_keys: If true, the keys are symbolized.

    • value_type: Defines the same type for all values.

Code Example:

config_param :key_values, :hash, default: {}, symbolize_keys: true, value_type: :string

def do_something
  value1 = @key_values[:key1]
  value2 = @key_values[:key2]
  # ...
end

Configuration Example:

key_values {"key1": "value1", "key2": "value2"} # written in JSON
key_values key1:value1,key2:value2

This configurations will be converted to:

{ key1: "value1", key2: "value2" }
PreviousHow to Write Tests for PluginNextUpgrade Plugin from v0.12

Last updated 3 years ago

Was this helpful?

If this article is incorrect or outdated, or omits critical information, please . is an open-source project under . All components are available under the Apache 2 License.

let us know
Fluentd
Cloud Native Computing Foundation (CNCF)