tcp - Fluentd

Search…
tcp
The in_tcp Input plugin enables Fluentd to accept TCP payload.
It is included in Fluentd's core.
Don't use this plugin for receiving logs from Fluentd client libraries. Use in_forward for such cases.
Example Configuration
1
<source>
2
  @type tcp
3
  tag tcp.events # required
4
  <parse>
5
    @type regexp
6
    expression /^(?<field1>\d+):(?<field2>\w+)$/
7
  </parse>
8
  port 20001   # optional. 5170 by default
9
  bind 0.0.0.0 # optional. 0.0.0.0 by default
10
  delimiter "\n" # optional. "\n" (newline) by default
11
</source>
Copied!
Example input:
1
$ echo '123456:awesome' | netcat 0.0.0.0 5170
Copied!
Parsed result:
1
{"field1":"123456","field2":"awesome"}
Copied!
Refer to the Configuration File article for the basic structure and syntax of the configuration file.
For <parse>, see Parse Section.
We have observed drastic performance improvements on Linux, with proper kernel parameter settings. If you have high-volume TCP traffic, follow Before Installing Fluentd instructions.
Plugin Helpers
​server​
​parser​
​extract​
​compat_parameters​
Parameters
See Common Parameters.
@type
The value must be tcp.
tag
type
default
version
string
required parameter
0.14.0
The tag of output events.
port
type
default
version
integer
5170
0.14.0
The port to listen to.
bind
type
default
version
string
0.0.0.0 (all addresses)
0.14.0
The bind address to listen to.
source_hostname_key
type
default
version
string
nil (no adding hostname)
0.14.10
The field name of the client's hostname. If set, the client's hostname will be set to its key. The default is nil (no adding hostname).
With this configuration:
1
source_hostname_key client_host
Copied!
The client's hostname is set to client_host field:
1
{
2
    ...
3
    "foo": "bar",
4
    "client_host": "client.hostname.org"
5
}
Copied!
source_address_key
type
default
version
string
nil (no adding source address)
1.4.2
The field name for the client's IP address. If set, Fluentd automatically adds the remote address to each data record.
For example, if you have the following configuration:
1
<source>
2
  @type tcp
3
  source_address_key client_addr
4
  # ...
5
</source>
Copied!
You will get something like below:
1
{
2
    ...
3
    "client_addr": "192.168.10.10"
4
    ...
5
}
Copied!
<transport> Section
type
default
available values
version
enum
tcp
tcp, tls
0.14.12
This section is for setting TLS transport or some general transport configurations.
General configuration
linger_timeout
type
default
available transport type
version
integer
0
tcp, tls
1.14.6
The timeout (seconds) to set SO_LINGER.
The default value 0 is to send RST rather than FIN to avoid lots of connections sitting in TIME_WAIT on closing on non-Windows.
You can set positive value to send FIN on closing on non-Windows.
(On Windows, Fluentd sends FIN when linger_timeout is 0 too).
1
<source>
2
  @type tcp
3
  # other plugin parameters
4
  <transport tcp>
5
    linger_timeout 1
6
  </transport>
7
</source>
Copied!
TLS configuration
1
<transport tls>
2
  cert_path /path/to/fluentd.crt
3
  # ...
4
</transport>
Copied!
See How to Enable TLS Encryption section for how to use and see Configuration Example for all supported parameters.
Without <transport tls>, in_tcp uses raw TCP.
<security> Section
required
multi
version
false
false
1.7.2
Adds <security>/<client> section to allow access by Host/IP/Network.
<client> Section
host
type
default
version
string
nil
1.7.2
The IP address or host name of the client.
This is exclusive with network.
network
type
default
version
string
nil
1.7.2
Network address specification.
This is exclusive with host.
<parse> Section
required
multi
version
true
false
0.14.10
in_tcp uses the parser plugin to parse the payload.
For more details:
​Parser Plugin Overview​
​Parse Section Configurations​
Code Example
Here is a Ruby example to send an event to in_tcp:
1
require 'socket'
2
​
3
# This example uses json payload.
4
# In in_tcp configuration, need to configure "@type json" in "<parse>"
5
TCPSocket.open('127.0.0.1', 5170) do |s|
6
  s.write('{"k":"v1"}' + "\n")
7
  s.write('{"k":"v2"}' + "\n")
8
end
Copied!
Tips
How to Enable TLS Encryption
in_tcp supports TLS transport.
Example:
1
<source>
2
  @type tcp
3
  port 5140
4
  bind 0.0.0.0
5
  <transport tls>
6
    ca_path /etc/pki/ca.pem
7
    cert_path /etc/pki/cert.pem
8
    private_key_path /etc/pki/key.pem
9
    private_key_passphrase PASSPHRASE
10
  </transport>
11
  tag tcp
12
</source>
Copied!
How to Enable TLS Mutual Authentication
Fluentd supports TLS mutual authentication (i.e. client certificate auth). If you want to use this feature, please set the client_cert_auth and ca_path options like this:
1
<source>
2
  @type tcp
3
  port 20001
4
  <transport tls>
5
    # ...
6
    client_cert_auth true
7
    ca_path /path/to/ca/cert
8
  </transport>
9
</source>
Copied!
When this feature is enabled, Fluentd will check all the incoming requests for a client certificate signed by the trusted CA. Requests with an invalid client certificate will fail.
To check if mutual authentication is working properly, issue these commands:
1
$ openssl s_client -connect localhost:20001 \
2
  -key path/to/client.key \
3
  -cert path/to/client.crt \
4
  -CAfile path/to/ca.crt
Copied!
If the connection gets established successfully, your setup is working fine.
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is an open-source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.