The csv parser plugin parses CSV format.

Parameters

See Parse section configurations​

keys

Names of fields included in each lines.

delimiter

The delimiter character (or string) of CSV values.

parser_type

The parser type used to parse log line.

• normal uses ruby's CSV.parse_line method.

• fast uses own light-weight implementation. This parser is several times faster than normal but it supports only typical patterns.

Here are fast supported CSV formats:

# non-quoted
value1,value2,value3,value4,value5
# quoted
"value1","val,ue2","va,lu,e3","val ue4",""
# escaped
"message","mes""sage","""message""",,""""""
# mixed
message,"mes,sage","me,ssa,ge",mess age,""

If your CSV format is not matched with above patterns, use normal parser instead.

Example

<parse>
  @type csv
  keys time,host,req_id,user
  time_key time
</parse>

With this configuration:

2013/02/28 12:00:00,192.168.0.1,111,-

This incoming event is parsed as:

time:
1362020400 (2013/02/28/ 12:00:00)
​
record:
{
  "host"   : "192.168.0.1",
  "req_id" : "111",
  "user"   : "-"
}

If you set null_value_pattern '-' in the configuration, user field becomes nil instead of "-".

If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.