All plugin types are subclass of Fluent::Plugin::Base
in Fluentd v0.14 or later. Base class has some features and methods which provides basic mechanism as plugins. This page shows these methods provided by Fluent::Plugin::Base
, and other methods provided commonly in some type of plugins.
The methods listed below are considered as public methods, and will be maintained not to break compatibility. Other methods may be changed without compatibility consideration.
require 'fluent/plugin/input' # this might be input, filter, output, parser, formatter, storage or buffer​module Fluent::Pluginclass MyExampleInput < Input# Plugins should be registered by calling ``Fluent::Plugin.register_TYPE`` method with name and ``self``.# The first argument String is to identify that plugin in configuration files.# The second argument is class of plugin itself, ``self`` in most cases.Fluent::Plugin.register_output('my_example', self)​desc "The port number"# config_param defines a parameter. You can refer a parameter via @port instance variable.# Without :default, a parameter is required.config_param :port, :integer​config_section :user, param_name: :users, multi: true, required: false dodesc "Username for authentication"config_param :username, :stringdesc "Password for authentication"config_param :password, :string, secret: trueend​def configure(conf)super​# If the configuration is invalid, raise Fluent::ConfigError.if @port <= 1024raise Fluent::ConfigError, "port number is too small: #{@port}"end​@users.each do |user|if user.password.length < 5raise Fluent::ConfigError, "password is too short for user '#{user.username}'"endendend​def startsuper# ...end​def shutdown# ...superendendend
Base class has some methods to create configurable parameters for plugins, or to provide system configurations to plugins.
Define a parameter.
name
: parameter name as symbol
type
: parameter type
options
: options for parameter
block
: if block is given, convert value via given block
About types and options, see Types of Configuration Parameters.
Code Example:
config_param :name, :string, default: "John Doe", alias: :full_name​config_param :pattern, do |value|Regexp.compile(value)end​config_param :delimiter, default: "\t" do |value|case valuewhen /SPACE/i then " "when /COMMA/i then ","else "\t"endend
Set default value to name
. If the default value has already been existed, raise ArgumentError
.
name
: The name of parameter
default_value
: Default value for the parameter
Code Example:
# lib/fluent/plugin/buffer.rbconfig_param :chunk_limit_size, :size, default: DEFAULT_CHUNK_LIMIT_SIZE​# lib/fluent/plugin/buf_file.rbconfig_set_default :chunk_limit_size, DEFAULT_CHUNK_LIMIT_SIZE
Set description to name
. If the description has already been existed, raise ArgumentError
. This is for internal API. Use desc
instead.
name
: The name of parameter
description
: Description about the parameter
Set description to immediately following parameter.
description
: Description about the parameter
Code Example:
desc "description about following parameter"config_param :user, :string
Define a section to construct structured configuration.
name
: The name of section
options
root
: If true, this section is root section. This is only for
internal use.
param_name
: This section name
final
: If true, subclass of this class cannot overwrite this
section. e.g. 3rd party plugins cannot overwrite buffer section
init
: If true, parameters in this section must have default
value. 3rd party plugins don't have to be conscious about this
option.
required
: If true, this section is required. Fluentd will
raise Fluent::ConfigError
when this section is missing in
configuration.
multi
: If true, users can write this section in configuration.
alias
: Alias of this section
Code Example:
config_section :user, multi: true doconfig_param :name, :stringconfig_param :password, :string, secret: trueend​# nested sectionsconfig_section :child, :param_name: "children", required: false, multi: true doconfig_param :name, :stringconfig_param :power, :integer, default: nilconfig_section :item, multi: true doconfig_param :name, :stringconfig_param :flavor, :stringendend
Configuration Example:
<user>name Alicepassword very-secret-password</user>​# nested sections<child>name Bobpower 1000<item>name potionflavor very sour orange</item><item>name gumiflavor super delicious apple</item></child>
Inherit section section_name
defined in super class.
section_name
: section name as Symbol
Returns Fluent::SystemConfig
instance.
For more details, see System Configuration.
Code Example:
def configure(conf)superroot_dir = system_config.root_dirend
Overwrite system config
This is for internal use and plugin testing.
For more details, see System Configuration.
options
: system configuration as Hash
Code Example:
test "plugin instance can overwrite system_config if needed" doplugin = ExampleInput.newplugin.configure(conf)# ...plugin.system_config_override("workers" => 3)# ...end
This method is used to initialize internal states (instance variables). Call super
if the plugin overrides this method.
Code Example:
def initializesuper@internal_counter = 0@in_memory_cache = nilend
conf
is an instance of Fluent::Config::Element
. Call super
if the plugin overrides this method. Fluentd's Configurable module (included in Base class) will traverse conf
object, and set values from configurations or default values into instance variables in super
. So instance variables or accessor methods are available after super
in #configure
method.
The code to configure the plugin by itself should be after super
.
Code Example:
def configure(conf)super​# cache_default_value is created/configured by config_param@in_memory_cache = Hash.new(@cache_default_value)end
The return value of this method will be ignored.
Returns Fluent::Log
instance
Log levels:
trace
debug
info
warn
error
fatal
For more details about Fluentd's logging mechanism, see Logging of Fluentd.
Code Example:
def start...log.info("This is info level log")# evaluate blog only when log level is tracelog.trace do# heavy calculation to trace plugin behavior"This is trace level log"endend
This method returns true or false, which indicates #router
method is provided or not. Default is false. If the plugin uses event_emitter
plugin helper, this method will return true.
Input plugin enables ``event_emitter`` plugin helper in default.
This method is called when Fluentd starts, after all configuration steps. Call super
if the plugin overrides this method.
Creating/Opening timers, threads, listening sockets, file handles and others should be done in this method after super
. Many of these may be provided as plugin helpers. See API details of each plugin helpers.
Code Example:
def startsuper​timer_execute(:my_example_timer, 30) do# any code which will be executed every 30 seconds# during Fluentd is runningendend
This method is called at first in shutdown sequence of Fluentd. Call super
if the plugin overrides this method.
This method should be used to enable flag to stop loops, network servers or something else, gracefully. This method SHOULD NOT do anything which may raise errors.
Code Example:
def start # examplesuper@my_thread_running = true@my_thread = thread_create(:example_code) doif @my_thread_runninglog.debug "loop is running"endendenddef stop@my_thread_running = false​superend
super
should be called at last in methods of shutdown sequence: stop
, before_shutdown
, shutdown
, after_shutdown
, close
and terminate
.
This method is called after #stop
and before #shutdown
. Call super
if the plugin overrides this method. There are no need to implement this method by 3rd party plugins in most cases. This method is used to control flushing buffered events in shutdown sequence.
This method is called when shutting down. Call super
if the plugin overrides this method.
The plugin should close file handles, network connections, listening servers and other resources. The plugin can emit events in this method, and cannot emit events after this method is called.
Code Example:
def shutdown@server.closerecords = my_convert_method(@server.rest_data)records.each do |record|router.emit(@tag, Fluent::Engine.now, record)end​superend
This method is called after #shutdown
. Call super
if the plugin overrides this method. There are no need to implement this method by 3rd party plugins in most cases. This method is used to control emitting events in shutdown sequence.
Call super
if the plugin overrides this method. The plugin can use this method to close resources which is used in plugins and cannot be closed in #shutdown
.
Call super
if the plugin overrides this method. The plugin can use this method to re-initialize internal states to make it possible to reuse plugin instances in tests or others.
The methods below are available in subclass of Input, Filter and Output. Other types
The plugin can include features of plugin helpers by calling this method with symbol arguments.
Code Example:
module Fluent::Pluginclass MyPlugin < InputFluent::Plugin.register_input('my_plugin', self)​# This call enables Timer and Storage plugin helpershelpers :timer, :storage​# ...endend
It is strongly recommended to call this method at the top of plugin class definition (just after calling #register_foo
) to show which plugin helpers this plugin uses explicitly.
This method provides an id string of plugin instance which is unique in Fluentd process. It might be specified by users in configuration files, or might be generated automatically. The plugin must not expect any fixed formats for return values.
This method returns true or false to indicate #plugin_id
is configured by users or not.
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.