How to Write Input Plugin
Last updated
Was this helpful?
Last updated
Was this helpful?
Extend Fluent::Plugin::Input
class and implement its methods.
See for details on the common APIs for all plugin types.
In most cases, input plugins start timers, threads, or network servers to listen on ports in #start
method and then call router.emit
in the callbacks of timers, threads or network servers to emit events.
Example:
To submit events, use router.emit(tag, time, record)
method, where:
tag
is a String
,
time
is the Fluent::EventTime
or Integer
as Unix time; and,
record
is a Hash
object.
Example:
To submit multiple events, use router.emit_stream(tag, es)
method, where:
tag
is a String
; and,
es
is a MultiEventStream
object.
Example:
Fluentd plugins assume the record is in JSON format so the key should be the String
, not Symbol
. If you emit a record with a key as Symbol
, it may cause a problem.
Example:
To do this, the following condition must be met:
This plugin can run in parallel with another Fluentd.
This is because there is a period when the old process and the new process run in parallel during a zero-downtime restart.
After addressing the following considerations and ensuring there are no issues, override this method. Then, the plugin will succeed with zero-downtime restart.
Handling Files
When handling files, there is a possibility of conflict.
Basically, input plugins that handle files should not support Zero-downtime restart.
Handling Sockets
When handling sockets on your own, be careful to avoid conflicts.
Fluentd input plugin has one or more points to be tested. Others aspects (parsing configurations, controlling buffers, retries, flushes, etc.) are controlled by the Fluentd core.
Fluentd also provides the test drivers for plugins. You can write tests for your own plugins very easily:
Testing for input plugins is mainly for:
Validation of configuration (i.e. #configure
)
Validation of the emitted events
To make testing easy, the test driver provides a dummy router, a logger and the functionality to override system and parser configurations, etc.
The lifecycle of plugin and test driver is:
Instantiate plugin driver which then instantiates the plugin
Configure plugin
Register conditions to stop/break running tests
Run test code (provided as a block for d.run
)
Assert results of tests by data provided by the driver
Test driver calls methods for plugin lifecycle at the beginning of Step # 4 (i.e. #start
) and at the end (i.e. #stop
, #shutdown
, etc.). It can be skipped by optional arguments of #run
.
For:
configuration tests, repeat steps # 1-2
full feature tests, repeat steps # 1-5
To support , you can override this method to return true
.
A socket provided as a shared socket by is shared between the old and new processes. So, such a plugin can support Zero-downtime restart.
See for details.
If this article is incorrect or outdated, or omits critical information, please . is an open-source project under . All components are available under the Apache 2 License.