syslog
Last updated
Last updated
The in_syslog Input plugin enables Fluentd to retrieve records via the syslog protocol on UDP or TCP.
It is included in Fluentd's core.
This tells Fluentd to create a socket listening on port 5140. You need to set up your syslog daemon to send messages to the socket. For example, if you're using rsyslogd, add the following lines to /etc/rsyslog.conf:
The retrieved data is organized as follows. Fluentd's tag is generated by the tag parameter (tag prefix), facility level, and priority. The record is parsed by the regexp here.
If you want to keep facility and priority in the record, set related parameters.
See Common Parameters.
@type (required)The value must be syslog.
tag (required)The prefix of the tag. The tag itself is generated by the tag prefix, facility level, and priority.
portinteger
5140
0.14.0
The port to listen to.
bindstring
0.0.0.0 (all addresses)
0.14.0
The bind address to listen to.
protocol_typeenum
udp
udp/tcp
0.14.0
The transport protocol used to receive logs. udp and tcp are supported.
This parameter is deprecated since v1.5. Use <transport> instead.
<transport> Sectionenum
udp
udp/tcp/tls
1.5.0
The protocol of the syslog transport.
This section is for setting TLS transport or some general transport configurations. See How to Enable TLS Encryption section for how to use and see Configuration Example and Config: Transport Section for all supported parameters.
message_length_limitsize
2048
0.14.2
The maximum length of a syslog message in bytes. If you send a larger message, change this parameter.
frame_typeenum
traditional
traditional/octet_count
1.3.0
Specifies the framing type in TCP protocol.
traditional
Messages are delimited by newline():
octet_count
Message has the message size prefix to delimit:
See also RFC 6587.
formatDeprecated parameter. Use <parse> instead.
<parse> DirectiveThe format of the log. This option is used to parse non-standard syslog formats using parser plugins.
Your <parse> regexp should not consider the 'priority' prefix of the log. For example, if in_syslog receives the log below:
Then, the format parser receives the following log:
If the <parse>/@type parameter is missing, the log data is assumed to have the canonical syslog format. It is same with the following configuration:
message_formatenum
rfc3164
rfc3164/rfc5424/auto
0.14.14
This parameter is used inside <parse> directive. The default is rfc3164.
Specifies the protocol format. Supported values are rfc3164, rfc5424 and auto. If your syslog uses rfc5424, use rfc5424 instead. Here is an example of message:
auto is useful when in_syslog receives both rfc3164 and rfc5424 message per source. in_syslog detects message format by using message prefix and parses it.
with_prioritybool
true
0.14.0
This parameter is used inside <parse> directive.
If with_priority is true, then syslog messages are assumed to be prefixed with a priority tag like <3>. This option exists since some syslog daemons output logs without the priority tag preceding the message body.
If you wish to parse syslog messages of arbitrary formats, in_tcp or in_udp are recommended.
emit_unmatched_linesbool
false
1.6.3
Emits unmatched lines when <parse> format is not matched for incoming logs.
Emitted record is {"unmatched_line" : "incoming line"} with ${tag parameter}.unmatched tag.
resolve_hostnamebool
nil
0.14.19
Tries to resolve hostname from IP addresses or not. Cannot set false when source_hostname_key is set.
send_keepalive_packetbool
false
1.14.0
Enables the TCP keepalive for sockets. See socket article for more details.
source_hostname_keystring
nil (no assign)
0.14.0
The field name of the client's hostname. If set, the client's hostname will be set to its key.
source_address_keystring
nil (no assign)
0.14.0
The field name of the client's address. If set, the client's address will be set to its key.
severity_keystring
nil (no assign)
1.7.3
The field name of the severity. If set, the severity will be set to its key.
If you set severity_key severity and got <6> started syslog message, severity field is info.
priority_keystring
nil (no assign)
0.14.10
This parameter is deprecated due to a misleading name. This sets severity, not priority.
This parameter will be removed in fluentd v2. Use severity_key instead.
facility_keystring
nil (no assign)
0.14.10
The field name of the facility. If set, the facility will be set to its key.
If you set facility_key facility and got <6> started syslog message, facility field is kern.
@log_levelThe @log_level option allows the user to set different levels of logging for each plugin. The supported log levels are: fatal, error, warn, info, debug, and trace.
Please see the logging article for further details.
This plugin assumes for delimiter character between syslog messages in one TCP connection by default. If you use syslog library in your application with <transport tcp>, add to your syslog message. See also rfc6587.
If your syslog uses octet counting mode, set frame_type octet_count in in_syslog configuration. See also frame_type parameter.
Since v1.5.0, in_syslog support TLS transport. Here is the configuration example with rsyslog:
in_syslog
rsyslog
If you use this plugin under the multi-process environment, the port will be shared.
With this configuration, 3 workers share 5140 port. No need of an additional port. The incoming data will be routed to the three (3) workers automatically.
First, check your message format follows RFC3164/RFC5424 or not. Some systems say RFC3164/RFC5424 but it sends non-RFC3164/RFC5424 message, e.g. invalid priority, different timestamp, lack/add fields.
If only timestamp is different, configure time_format in <parse> may help.
If other parts are different, the syslog parser cannot parse your message. To resolve the problem, there are several approaches:
Use regex parser or write your parser
Use in_udp/in_tcp with other parsers
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is an open-source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.
