Versions | v0.12 (td-agent2) | v0.10 (td-agent1)

Secure Forward Input Plugin

The in_secure_forward input plugin accepts messages via SSL with authentication (cf. out_secure_forward).

This document doesn't describe all parameters. If you want to know full features, check the Further Reading section.

Table of Contents

Installation

in_secure_forward is not included in either td-agent package or fluentd gem. In order to install it, please refer to the Plugin Management article.

Example Configurations

This section provides some example configurations for in_secure_forward.

Minimalist Configuration

At first, generate private CA file by secure-forward-ca-generate, then copy that file to output plugin side by safe way (scp, or anyway else).

<source>
  @type secure_forward
  shared_key      secret_string
  self_hostname   server.fqdn.local  # This fqdn is used as CN (Common Name) of certificates
  secure true
  ca_cert_path        /path/to/certificate/ca_cert.pem
  ca_private_key_path /path/to/certificate/ca_key.pem
  ca_private_key_passphrase passphrase_for_private_CA_secret_key
</source>

Check username/password from Clients

<source>
  @type secure_forward
  shared_key         secret_string
  self_hostname      server.fqdn.local
  secure true
  ca_cert_path        /path/to/certificate/ca_cert.pem
  ca_private_key_path /path/to/certificate/ca_key.pem
  ca_private_key_passphrase passphrase_for_private_CA_secret_key
  authentication     yes # Deny clients without valid username/password
  <user>
    username tagomoris
    password foobar012
  </user>
  <user>
    username frsyuki
    password yakiniku
  </user>
</source>

Deny Unknown Source IP/hosts

<source>
  @type secure_forward
  shared_key         secret_string
  self_hostname      server.fqdn.local
  secure true
  ca_cert_path        /path/to/certificate/ca_cert.pem
  ca_private_key_path /path/to/certificate/ca_key.pem
  ca_private_key_passphrase passphrase_for_private_CA_secret_key
  allow_anonymous_source no  # Allow to accept from nodes of <client>
  <client>
    host 192.168.10.30
    # network address (ex: 192.168.10.0/24) NOT Supported now
  </client>
  <client>
    host your.host.fqdn.local
    # wildcard (ex: *.host.fqdn.local) NOT Supported now
  </client>
</source>

You can use the username/password check and client check together:

<source>
  @type secure_forward
  shared_key         secret_string
  self_hostname      server.fqdn.local
  secure true
  ca_cert_path        /path/to/certificate/ca_cert.pem
  ca_private_key_path /path/to/certificate/ca_key.pem
  ca_private_key_passphrase passphrase_for_private_CA_secret_key
  allow_anonymous_source no  # Allow to accept from nodes of <client>
  authentication         yes # Deny clients without valid username/password
  <user>
    username tagomoris
    password foobar012
  </user>
  <user>
    username frsyuki
    password sukiyaki
  </user>
  <user>
    username repeatedly
    password sushi
  </user
  <client>
    host 192.168.10.30      # allow all users to connect from 192.168.10.30
  </client>
  <client>
    host  192.168.10.31
    users tagomoris,frsyuki # deny repeatedly from 192.168.10.31
  </client>
  <client>
    host 192.168.10.32
    shared_key less_secret_string # limited shared_key for 192.168.10.32
    users      repeatedly         # and repeatedly only
  </client>
</source>

Secure Sender-Receiver Setup

Please refer to the Secure Sender-Receiver Setup sample documentation.

Parameters

type

This parameter is required. Its value must be secure_forward.

port (integer)

The default value is 24284.

bind (string)

The default value is 0.0.0.0.

secure (bool)

Indicate published connection is secure or not. Specify yes (or true) if secure encryption needed.

self_hostname (string)

Default value of the auto-generated certificate common name (CN).

shared_key (string)

Optional shared key.

allow_keepalive (bool)

Accept keepalive connection. The default value is true.

allow_anonymous_source (bool)

Accept connections from unknown hosts.

authentication (bool)

Require password authentication. The default value is false.

ca_cert_path (string)

The path to the private CA certificate file, which is required to use private CA. (One of this parameter or cert_path is required for secure yes configuration.)

ca_private_key_path (string)

The path to the private key for private CA certificate key file.

ca_private_key_passphrase (string)

The passphrase string for private key file, specified by ca_private_key_path.

read_length (size)

The number of bytes read per nonblocking read. The default value is 8MB=810241024 bytes.

read_interval_msec (integer)

The interval between the non-blocking reads, in milliseconds. The default value is 50.

socket_interval_msec (integer)

The interval between SSL reconnects in milliseconds. The default value is 200.

Buffered Output Parameters

For advanced usage, you can tune Fluentd’s internal buffering mechanism with these parameters.

buffer_type

The buffer type is memory by default (buf_memory) for the ease of testing, however file (buf_file) buffer type is always recommended for the production deployments. If you use file buffer type, buffer_path parameter is required.

buffer_queue_limit, buffer_chunk_limit

The length of the chunk queue and the size of each chunk, respectively. Please see the Buffer Plugin Overview article for the basic buffer structure. The default values are 64 and 8m, respectively. The suffixes “k” (KB), “m” (MB), and “g” (GB) can be used for buffer_chunk_limit.

flush_interval

The interval between data flushes. The default is 60s. The suffixes “s” (seconds), “m” (minutes), and “h” (hours) can be used.

flush_at_shutdown

If set to true, Fluentd waits for the buffer to flush at shutdown. By default, it is set to true for Memory Buffer and false for File Buffer.

retry_wait, max_retry_wait

The initial and maximum intervals between write retries. The default values are 1.0 seconds and unset (no limit). The interval doubles (with +/-12.5% randomness) every retry until max_retry_wait is reached. In the default configuration the last retry waits for approximately 131072 sec, roughly 36 hours.

retry_limit, disable_retry_limit

The limit on the number of retries before buffered data is discarded, and an option to disable that limit (if true, the value of retry_limit is ignored and there is no limit). The default values are 17 and false (not disabled). If the limit is reached, buffered data is discarded and the retry interval is reset to its initial value (retry_wait).

num_threads

The number of threads to flush the buffer. This option can be used to parallelize writes into the output(s) designated by the output plugin. Increasing the number of threads improves the flush throughput to hide write / network latency. The default is 1.

slow_flush_log_threshold

The threshold for checking chunk flush performance. The default value is 20.0 seconds. Note that parameter type is float, not time.

If chunk flush takes longer time than this threshold, fluentd logs warning message like below:

2016-12-19 12:00:00 +0000 [warn]: buffer flush took longer time than slow_flush_log_threshold: elapsed_time = 15.0031226690043695 slow_flush_log_threshold=10.0 plugin_id="foo"

log_level option

The log_level option allows the user to set different levels of logging for each plugin. The supported log levels are: fatal, error, warn, info, debug, and trace.

Please see the logging article for further details.

Further Reading

Last updated: 2015-12-01 21:20:32 UTC

Versions | v0.12 (td-agent2) | v0.10 (td-agent1)

If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF), originally invented by Treasure Data, Inc. All components are available under the Apache 2 License.

Interested in the Fluentd Newsletters?