csv

The csv parser plugin parses CSV format.

Parameters

See Parse section configurations

keys

type

default

version

array of string

required parameter

0.14.9

Names of fields included in each lines.

delimiter

type

default

version

string

,

0.14.2

The delimiter character (or string) of CSV values.

parser_type

type

default

available values

version

enum

normal

normal/fast

1.7.0

The parser type used to parse log line.

  • normal uses ruby's CSV.parse_line method.

  • fast uses own light-weight implementation. This parser is several times faster than normal but it supports only typical patterns.

Here are fast supported CSV formats:

# non-quoted
value1,value2,value3,value4,value5
# quoted
"value1","val,ue2","va,lu,e3","val ue4",""
# escaped
"message","mes""sage","""message""",,""""""
# mixed
message,"mes,sage","me,ssa,ge",mess age,""

If your CSV format is not matched with above patterns, use normal parser instead.

Example

<parse>
@type csv
keys time,host,req_id,user
time_key time
</parse>

With this configuration:

2013/02/28 12:00:00,192.168.0.1,111,-

This incoming event is parsed as:

time:
1362020400 (2013/02/28/ 12:00:00)
record:
{
"host" : "192.168.0.1",
"req_id" : "111",
"user" : "-"
}

If you set null_value_pattern '-' in the configuration, user field becomes nil instead of "-".

If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.