The out_exec
TimeSliced Output plugin passes events to an external program. The program receives the path to a file containing the incoming events as its last argument. The file format is tab-separated values (TSV) by default.
It is included in Fluentd's core.
<match pattern>@type execcommand cmd arg arg<format>@type tsvkeys k1,k2,k3</format><inject>tag_key k1time_key k2time_format %Y-%m-%d %H:%M:%S</inject></match>
Please see the Configuration File article for the basic structure and syntax of the configuration file.
This example illustrates how to run FizzBuzz with out_exec
.
We assume that the input file is specified by the last argument in the command line (ARGV[-1]
). The following script fizzbuzz.py
runs FizzBuzz against the new-line delimited sequence of natural numbers (1, 2, 3...) and writes the output to foobar.out
:
#!/usr/bin/env pythonimport sysinput = file(sys.argv[-1])output = file("foobar.out", "a")for line in input:fizzbuzz = int(line.split("\t")[0])s = ''if fizzbuzz%3 == 0:s += 'fizz'if fizzbuzz%5 == 0:s += 'buzz'if len(s) > 0:output.write(s+"\n")else:output.write(str(fizzbuzz)+"\n")output.close
Note that this program is written in Python. For out_exec
(as well as out_exec_filter
and in_exec
), the program can be written in any language, not just Ruby.
Then, configure Fluentd as follows:
<source>@type forward</source><match fizzbuzz>@type execcommand python /path/to/fizzbuzz.py<buffer>@type filepath /path/to/buffer_pathflush_interval 5s # for debugging/checking</buffer><format>@type tsvkeys fizzbuzz</format></match>
The @type tsv
and keys fizzbuzz
in <format>
tells Fluentd to extract the fizzbuzz
field and output it as TSV. This simple example has a single key, but you can of course extract multiple fields and use format json
to output newline-delimited JSON.
The intermediate TSV is at /path/to/buffer_path
, and the command python /path/to/fizzbuzz.py /path/to/buffer_path
is run. This is why in fizzbuzz.py
, it is reading the file at sys.argv[-1]
.
If you start Fluentd and run this command:
$ for i in `seq 15`; do echo "{\"fizzbuzz\":$i}" | fluent-cat fizzbuzz; done
Then, after 5 seconds, you get a file named foobar.out
:
$ cat foobar.out12fizz4buzzfizz78fizzbuzz11fizz1314fizzbuzz
Asynchronous
See Output Plugin Overview for more details.
​inject
​
​formatter
​
​compat_parameters
​
​child_process
​
​Common Parameters​
The value must be exec
.
type | default | version |
string | Nothing | 0.14.0 |
The command (program) to execute. The exec
plugin passes the path of flushed buffer chunk as the last argument.
If you set command
parameter like this:
command cmd arg arg
The actual command execution is:
cmd arg arg /path/to/file
If cmd
does not exist in PATH, you need to specify the absolute path, e.g. /path/to/cmd
.
type | default | version |
time | 270 | 0.14.9 |
Command (program) execution timeout.
See Format Section for more details.
type | default | version |
string | tsv | 0.14.9 |
The format used to map the incoming events to the program input.
Overwrites the default value in this plugin.
See Inject Section for more details.
type | default | version |
string | string | 0.14.9 |
Overwrites the default value in this plugin.
type | default | version |
bool | false | 0.14.9 |
Overwrites the default value in this plugin.
See Buffer Section for more details.
type | default | version |
time | 300 | 0.14.9 |
Overwrites the default value in this plugin.
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is an open-source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.