Versions | v0.14 (td-agent3)

This page is for v0.14, not the latest stable version which is v0.12. For the latest stable version of this article, click here.


Plugin Base API

All plugin types are subclass of Fluent::Plugin::Base in Fluentd v0.14 or later. Base class has some features and methods which provides basic mechanism as plugins. This page shows these methods provided by Fluent::Plugin::Base, and other methods provided commonly in some type of plugins.

The methods listed below are considered as public methods, and will be maintained not to break compatibility. Other methods may be changed without compatibility consideration.

require 'fluent/plugin/input' # this might be input, filter, output, parser, formatter, storage or buffer

module Fluent::Plugin
  class MyExampleInput < Input
    # Plugins should be registered by calling ``Fluent::Plugin.register_TYPE`` method with name and ``self``.
    # The first argument String is to identify that plugin in configuration files.
    # The second argument is class of plugin itself, ``self`` in most cases.
    Fluent::Plugin.register_output('my_example', self)

    # config_param defines a parameter. You can refer a parameter via @port instance variable.
    # Without :default, a parameter is required.
    config_param :port, :integer

    config_section :user, param_name: :users, multi: true, required: false do
      config_param :username, :string
      config_param :password, :string
    end

    def configure(conf)
      super

      # If the configuration is invalid, raise Fluent::ConfigError.
      if @port <= 1024
        raise Fluent::ConfigError, "port number is too small: #{@port}"
      end

      @users.each do |user|
        if user.password.length < 5
          raise Fluent::ConfigError, "password is too short for user '#{user.username}'"
        end
      end
    end

    def start
      super
      # ...
    end

    def shutdown
      # ...
      super
    end
  end
end

Table of Contents

Class methods

Base class has some methods to create configurable parameters for plugins, or to provide system configurations to plugins.

.config_param(name, type = nil, **options, &block)

TODO: write this section later

About types, see Types of Configuration Parameters.

.config_set_default(name, default_value)

TODO: write this section later

.config_set_desc(name, description)

TODO: write this section later

.desc(description)

TODO: write this section later

.config_section(name, **options, &block)

TODO: write this section later

.configured_in(section_name)

TODO: write this section later

.system_config

TODO: write this section later

.system_config_override(options = {})

TODO: write this section later

Instance methods

#initialize

This method is used to initialize internal states (instance variables). Call super if the plugin overrides this method.

def initialize
  super
  @internal_counter = 0
  @in_memory_cache = nil
end

#configure(conf)

conf is an instance of Fluent::Config::Element. Call super if the plugin overrides this method. Fluentd’s Configurable module (included in Base class) will traverse conf object, and set values from configurations or default values into instance variables in super. So instance variables or accessor methods are available after super in #configure method.

The code to configure the plugin by itself should be after super.

def configure(conf)
  super

  # cache_default_value is created/configured by config_param
  @in_memory_cache = Hash.new(@cache_default_value)
end

The return value of this method will be ignored.

#log

TODO: write this section later with list of logger methods

#has_router?

This method returns true or false, which indicates #router method is provided or not. Default is false. If the plugin uses event_emitter plugin helper, this method will return true.

Input plugin enables ``event_emitter`` plugin helper in default.

#start

This method is called when Fluentd starts, after all configuration steps. Call super if the plugin overrides this method.

Creating/Opening timers, threads, listening sockets, file handles and others should be done in this method after super. Many of these may be provided as plugin helpers. See API details of each plugin helpers.

def start
  super

  timer_execute(:my_example_timer, 30) do
    # any code which will be executed every 30 seconds
    # during Fluentd is running
  end
end

#stop

This method is called at first in shutdown sequence of Fluentd. Call super if the plugin overrides this method.

This method should be used to enable flag to stop loops, network servers or something else, gracefully. This method SHOULD NOT do anything which may raise errors.

def start # example
  super
  @my_thread_running = true
  @my_thread = thread_create(:example_code) do
    if @my_thread_running
      log.debug "loop is running"
    end
  end
end
def stop
  @my_thread_running = false

  super
end

super should be called at last in methods of shutdown sequence: stop, before_shutdown, shutdown, after_shutdown, close and terminate.

#before_shutdown

This method is called after #stop and before #shutdown. Call super if the plugin overrides this method. There are no need to implement this method by 3rd party plugins in most cases. This method is used to control flushing buffered events in shutdown sequence.

#shutdown

This method is called when shutting down. Call super if the plugin overrides this method.

The plugin should close file handles, network connections, listening servers and other resources. The plugin can emit events in this method, and cannot emit events after this method is called.

def shutdown
  @server.close
  records = my_convert_method(@server.rest_data)
  records.each do |record|
    router.emit(@tag, Fluent::Engine.now, record)
  end

  super
end

#after_shutdown

This method is called after #shutdown. Call super if the plugin overrides this method. There are no need to implement this method by 3rd party plugins in most cases. This method is used to control emitting events in shutdown sequence.

#close

Call super if the plugin overrides this method. The plugin can use this method to close resources which is used in plugins and cannot be closed in #shutdown.

#terminate

Call super if the plugin overrides this method. The plugin can use this method to re-initialize internal states to make it possible to reuse plugin instances in tests or others.

Methods for Input/Filter/Output

The methods below are available in subclass of Input, Filter and Output. Other types

.helpers(*symbols)

The plugin can include features of plugin helpers by calling this method with symbol arguments.

module Fluent::Plugin
  class MyPlugin < Input
    Fluent::Plugin.register_input('my_plugin', self)

    # This call enables Timer and Storage plugin helpers
    helpers :timer, :storage

    # ...
  end
end

It is strongly recommended to call this method at the top of plugin class definition (just after calling #register_foo) to show which plugin helpers this plugin uses explicitly.

#plugin_id

This method provides an id string of plugin instance which is unique in Fluentd process. It might be specified by users in configuration files, or might be generated automatically. The plugin must not expect any fixed formats for return values.

#plugin_id_configured?

This method returns true or false to indicate #plugin_id is configured by users or not.

Last updated: 2016-06-13 06:11:23 UTC

Versions | v0.14 (td-agent3)

If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF), originally invented by Treasure Data, Inc. All components are available under the Apache 2 License.

Interested in the Fluentd Newsletters?