How to Write Tests for Plugin
This article explains how to write Fluentd plugin test code using test-unit
.
You can write test code with any other testing framework such as RSpec
, minitest
, etc.
Basics of Plugin Testing
Fluentd provides useful Test Drivers according to plugin type. We can write maintainable test code for plugins using them. We can write helper.rb for output plugin as follows:
Note that Fluentd provides useful Test Drivers for input
, output
, filter
, parser
and formatter
.
The recommended fluentd plugin project structure is:
Plugin Test Driver Overview
There are some useful Test Drivers for plugin testing. We can write test code for plugins as following:
Testing Utility Methods
You can get a plugin instance by calling Test Driver instance's #instance
method. If utility methods are private
, use __send__
.
Test Driver Base API
The methods in this section are available for all Test Driver.
initialize(klass, opts: {}, &block)
initialize(klass, opts: {}, &block)
Initializes Test Driver instance.
klass
: A class of Fluentd pluginopts
: Overwrite system config. This parameter is useful for testing multiworkers.
block
: Customize plugin behavior. We can overwrite plugin code in thisblock.
Example:
configure(conf, syntax: :v1)
configure(conf, syntax: :v1)
Configures plugin instance managed by the Test Driver.
conf
:Fluent::Config::Element
orString
syntax
: {:v1
,:v0
,:ruby
}:v0
is obsolete.
Example:
end_if(&block)
end_if(&block)
Registers the conditions to stop the running Test Driver gracefully.
All registered conditions must be true
before Test Driver stops.
break_if(&block)
break_if(&block)
Registers the conditions to stop the running Test Driver.
Test Driver should stop running if some of the breaking conditions are true
.
broken?
broken?
Returns true
when some of the breaking conditions are true
. Otherwise false
.
run(timeout: nil, start: true, shutdown: true, &block)
run(timeout: nil, start: true, shutdown: true, &block)
Runs the Test Driver. This Test Driver will stop running immediately after evaluating the block
if given.
Otherwise, you must register the conditions to stop the running Test Driver.
This method may be overridden in subclasses.
timeout
: timeout (seconds)start
: iftrue
, start the Test Driver. Otherwise, invokeinstance_start
method to start it.
shutdown
: iftrue
, shut down the running Test Driver.
Example:
stop?
stop?
Returns true
when all the stopping conditions are true
. Otherwise false
.
logs
logs
Returns logs managed by this Test Driver.
instance
instance
Returns the plugin instance managed by this Test Driver.
Test Driver Base Owner API
filter
output
multi_output
run(expect_emits: nil, expect_records: nil, timeout: nil, start: true, shutdown: true, &block)
run(expect_emits: nil, expect_records: nil, timeout: nil, start: true, shutdown: true, &block)
Run Test Driver. This Test Driver will stop running immediately after evaluating block
if given.
Otherwise, you must register conditions to stop running Test Driver.
This method may be overridden in subclasses.
expect_emits
: set the number of expected emitsexpect_records
: set the number of expected recordstimeout
: timeout (seconds)start
: iftrue
, start the Test Driver. Otherwise, invoke theinstance_start
method to start it.shutdown
: iftrue
, shut down the running Test Driver.
Example:
events(tag: nil)
events(tag: nil)
Returns the events filtered by the given tag.
tag
: filter by this tag. If omitted, it returns all the events.
event_streams(tag: nil)
event_streams(tag: nil)
Returns the event streams filtered by the given tag.
tag
: filter by this tag. If omitted, it returns all the event streams.
emit_count
emit_count
Returns the number of invoking router.emit
.
If you want to delay the stopping of the Test Driver until a certain number of records are emitted, you can use d.run(expected_records: n)
instead.
Example:
record_count
record_count
Returns the number of records.
If you want to delay the stopping of the Test Driver until a certain number of records are emitted, you can use d.run(expected_records: n)
instead.
error_events(tag: nil)
error_events(tag: nil)
Returns error events filtered by the given tag.
tag
: filter by this tag. If omitted, it returns all error events.
Test Driver Base owned API
configure(conf, syntax: :v1)
configure(conf, syntax: :v1)
Configures plugin instance managed by this Test Driver.
conf
:Fluent::Config::Element
,String
, orHash
syntax
: Supported: {:v1
,:v0
,:ruby
}:v0
is obsolete.
Test Driver Event Feeder API
filter
output
multi_output
run(default_tag: nil, **kwargs, &block)
run(default_tag: nil, **kwargs, &block)
Runs EventFeeder
.
default_tag
: the default tag of the event
Example:
feed(tag, time, record)
feed(tag, time, record)
Feeds an event to plugin instance.
tag
: the tag of the eventtime
: event timestamprecord
: event record
Example:
feed(tag, array_event_stream)
feed(tag, array_event_stream)
Feeds an array of event stream to plugin instance.
tag
: the tag of the eventarray_event_stream
: array of[time, record]
time
: event timestamprecord
: event record
Example:
feed(tag, es)
feed(tag, es)
Feeds an event stream to plugin instance.
tag
: the tag of the eventes
: event stream object
Example:
feed(record)
feed(record)
Feeds an event with default tag to plugin instance.
record
: event record
Example:
feed(time, record)
feed(time, record)
Feeds an event with default tag to plugin instance.
time
: event timestamprecord
: event record
Example:
feed(array_event_stream)
feed(array_event_stream)
Feeds an array of event stream with default tag to plugin instance.
array_event_stream
: array of[time, record]
time
: event timestamprecord
: event record
Example:
feed(es)
feed(es)
Feeds an event stream with default tag to plugin instance.
es
: event stream object
Example:
Test Driver Filter API
filtered_records
filtered_records
Collects the filtered records.
Test Driver Output API
run(flush: true, wait_flush_completion: true, force_flush_retry: false, **kwargs, &block)
run(flush: true, wait_flush_completion: true, force_flush_retry: false, **kwargs, &block)
flush
: flush forciblywait_flush_completion
: iftrue
, waiting forflush
to completeforce_flush_retry
: iftrue
, retrying flush forcibly
Run Test Driver. This Test Driver will be stop running immediately after evaluating the block
if given.
Otherwise, you must register conditions to stop running Test Driver.
Example:
formatted
formatted
Returns the formatted records.
Example:
flush
flush
Flushes forcibly.
Example:
Test Helpers
assert_equal_event_time(expected, actual, message = nil)
assert_equal_event_time(expected, actual, message = nil)
Asserts the EventTime
instance.
expected
: expectedEventTime
instanceactual
: actualEventTime
instancemessage
: message to display when assertion fails
Example:
config_element(name = 'test', argument = '', params = {}, elements = [])
config_element(name = 'test', argument = '', params = {}, elements = [])
Create Fluent::Config::Element
instance.
name
: element name such asmatch
,filter
,source
,buffer
,inject
,format
,parse
, etc.argument
: argument for section defined byconfig_argument
params
: parameters for section defined byconfig_element
elements
: child elements of this config element
Example:
event_time(str = nil, format: nil)
event_time(str = nil, format: nil)
Creates a Fluent::EventTime
instance.
str
: time represented asString
format
: parsestr
astime
according to this format. See also
Example:
with_timezone(tz, &block)
with_timezone(tz, &block)
Processes the given block with tz
. This method overrides ENV['TZ']
while processing its block
.
tz
: timezone. This is set toENV['TZ']
.
Example:
with_worker_config(root_dir: nil, workers: nil, worker_id: nil, &block)
with_worker_config(root_dir: nil, workers: nil, worker_id: nil, &block)
Processes block
with the given parameters. This method overrides the system configuration while processing its block
.
This is useful for testing Fluentd's internal behavior related to multi workers.
root_dir
: root directoryworkers
: the number of workersworker_id
: ID of workers
Example:
time2str(time, localtime: false, format: nil)
time2str(time, localtime: false, format: nil)
Converts time
to String
.
This is useful for testing the formatter.
time
:Fluent::EventTime
instance. See alsolocaltime
: Iftrue
, processestime
aslocaltime
. Otherwise UTC.format
: See also
msgpack(type)
msgpack(type)
type
: Available types: {:factory
,:packer
,:unpacker
}
Shorthand for:
Fluent::MessagePackFactory.factory
Fluent::MessagePackFactory.packer
Fluent::MessagePackFactory.unpacker
Example:
capture_stdout(&block)
capture_stdout(&block)
Captures the standard output while processing the given block.
This is useful for testing Fluentd utility commands.
Example:
Testing Input Plugins
You must test the input plugins' router#emit
method. But you do not have to test this method explicitly. Its testing code pattern is encapsulated in the Input Test Driver.
You can write input plugins test like this:
Testing Filter Plugins
You must test filter plugins' #filter
method. But you do not have to test this method explicitly. Its testing code pattern is encapsulated in Filter Test Driver.
You can write filter plugins test like this:
Testing Output Plugins
You must test output plugins' #process
or #write
or #try_write
method. But you do not have to test this method explicitly. Its testing code pattern is encapsulated in the Output Test Driver.
You can write output plugins test like this:
Testing Parser Plugins
You must test the parser plugins' #parse
method.
You can write parser plugins test like this:
Testing Formatter Plugins
You must test the formatter plugins' #format
method.
You can write formatter plugins test like this:
Tests for Logs
Testing logs is easy.
Code example:
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is an open-source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.
Last updated