forward
Last updated
Last updated
The out_forward Buffered Output plugin forwards events to other fluentd nodes. This plugin supports load-balancing and automatic fail-over (i.e. active-active backup). For replication, please use out_copy plugin.
The out_forward plugin detects server faults using a φ accrual failure detector algorithm. You can customize the parameters of the algorithm. When a server fault recovers, the plugin makes the server available automatically after a few seconds.
The out_forward plugin supports at-most-once and at-least-once semantics. The default is at-most-once.
It is included in Fluentd's core.
Please see the Configuration File article for the basic structure and syntax of the configuration file.
Synchronous
Asynchronous
See Output Plugin Overview for more details.
@typeThe value must be forward.
<server> (at least one is required)true
true
0.14.5
The destination servers. Each server has the following parameters:
host
name
port
shared_key
username
password
standby
weight
If you want to manage destination servers by flexible approach, use <service_discovery> instead.
hoststring
required parameter
0.14.5
The IP address or host name of the server.
namestring
nil
0.14.5
The name of the server. Used for logging and certificate verification in TLS transport (when the host is the address).
portinteger
24224
0.14.5
The port number of the host. Note that both TCP packets (event stream) and UDP packets (heartbeat messages) are sent to this port.
shared_keystring
nil
0.14.5
The shared key per server.
usernamestring
"" (empty string)
0.14.5
The username for authentication.
passwordstring
"" (empty string)
0.14.5
The password for authentication.
standbybool
false
0.14.5
Marks a node as the standby node for an Active-Standby model between Fluentd nodes. When an active node goes down, the standby node is promoted to an active node. The standby node is not used by the out_forward plugin until then.
weightinteger
60
0.14.5
The load balancing weight. If the weight of one server is 20 and the weight of the other server is 30, events are sent in a 2:3 ratio. The default weight is 60.
<service_discovery>Use service discovery plugin instead of fixed <server> list. See also Service Discovery Plugin Overview for more details.
require_ack_responsebool
false
0.14.0
Changes the protocol to at-least-once. The plugin waits the ack from destination's in_forward plugin.
ack_response_timeouttime
190
0.14.0
This option is used when require_ack_response is true. This default value is based on popular tcp_syn_retries.
If set 0, this plugin does not wait for the ack response.
send_timeouttime
60
0.14.0
The timeout time when sending event logs.
connect_timeouttime
nil(no timeout)
1.6.0
The connection timeout for the socket. When the connection is timed out during the connection establishment, Errno::ETIMEDOUT error is raised.
recover_waittime
10
0.14.0
The wait time before accepting a server fault recovery.
heartbeat_typeenum
transport
transport, tcp, udp, none
0.14.12
Specifies the transport protocol for heartbeats. Set none to disable.
heartbeat_intervaltime
1
0.14.0
The interval of the heartbeat packer.
phi_failure_detectorbool
true
0.14.0
Use the "Phi accrual failure detector" to detect server failure.
phi_thresholdinteger
16
0.14.0
The threshold parameter used to detect server faults.
phi_threshold is directly related to heartbeat_interval. If you are using longer heartbeat_interval, please use the larger phi_threshold. Otherwise, you will see frequent detachments of destination servers. The default value 16 is tuned for heartbeat_interval 1s.
hard_timeouttime
60
0.14.0
The hard timeout used to detect server failure. The default value is equal to the send_timeout parameter.
expire_dns_cachetime
nil (persistent cache)
0.14.0
Sets TTL to expire DNS cache in seconds. Set 0 not to use DNS Cache.
dns_round_robinbool
false
0.14.0
Enable client-side DNS round robin. Uniform randomly pick an IP address to send data when a hostname has several IP addresses.
heartbeat_type udp is not available with dns_round_robintrue. Use heartbeat_type tcp or heartbeat_type none.
ignore_network_errors_at_startupbool
false
0.14.12
Ignores DNS resolution and errors at startup time.
tls_versionenum
TLSv1_2
TLSv1_1, TLSv1_2
0.14.12
The default version of TLS transport.
tls_ciphersstring
ALL:!aNULL:!eNULL:!SSLv2 (OpenSSL > 1.0.0 default)
0.14.12
The cipher configuration of TLS transport.
tls_insecure_modebool
false
0.14.12
Skips all verification of certificates or not.
tls_allow_self_signed_certbool
false
0.14.12
Allows self-signed certificates or not.
tls_verify_hostnamebool
true
0.14.12
Verifies hostname of servers and certificates or not in TLS transport.
If the following conditions are met, you must set tls_verify_hostname false explicitly to forward events correctly:
specify host in <server> section with IP address, not hostname
specify server certificate file with tls_cert_path which contains common name (CN) field with IP address, not hostname
tls_cert_patharray of string
nil
0.14.12
The additional CA certificate path for TLS.
tls_client_cert_pathstring
nil
1.3.2
The client certificate path for TLS.
tls_client_private_key_pathstring
nil
1.3.2
The client private key path for TLS.
tls_client_private_key_passphrasestring
nil
1.3.2
The TLS private key passphrase for the client.
tls_cert_thumbprintstring
nil
1.7.1
The certificate thumbprint for searching from Windows system certstore. This parameter is for Windows only.
tls_cert_logical_store_namestring
nil
1.7.1
The certificate logical store name on Windows system certstore. This parameter is for Windows only.
tls_cert_use_enterprise_storestring
true
1.7.1
Enables the certificate enterprise store on Windows system certstore. This parameter is for Windows only.
keepalivebool
false
1.5.0
Enables the keepalive connection.
keepalive_timeouttime
nil
1.5.0
Timeout for keepalive. Default value is nil which means to keep the connection alive as long as possible.
<security> Sectionfalse
false
0.14.5
This section contains parameters related to authentication:
self_hostname
shared_key
self_hostnamestring
required parameter
0.14.5
The hostname.
shared_keystring
required parameter
0.14.5
The shared key for authentication. If you want to specify shared_key for specific server, use the <server> section.
<secondary>false
false
0.14.0
The backup destination that is used when all servers are unavailable.
For more details, see Secondary Output.
verify_connection_at_startupbool
false
1.3.1
Verify that a connection can be made with one of out_forward nodes at the time of startup.
<buffer>See Buffer Section Configurations for more details.
chunk_keysarray
tag
0.14.9
Overwrites the default value in this plugin.
If you have set up TLS/SSL encryption for the receiving server, you need to tell the output forwarder to use encryption by setting the transport parameter:
If you are using a self-signed certificate, copy the certificate file to the forwarding server, then add the following settings:
After updating the settings, please confirm that the forwarded data is being received by the destination node properly.
If you have set up TLS/SSL encryption in the receiving server, you need to tell the output forwarder to use encryption by setting the transport parameter.
Valid logical store names are:
MY
CA
ROOT
AUTHROOT
DISALLOWED
SPC
TRUST
TRUSTEDPEOPLE
TRUSTEDPUBLISHER
CLIENTAUTHISSUER
TRUSTEDDEVICES
SMARTCARDROOT
WEBHOSTING
REMOTE DESKTOP
Logical store name is case-insensitive. Note that this section configurations work only for Windows.
If you are using a self-signed certificate, export the certificate file from Windows Certstore and copy to the forwarding server, then add the following settings:
Note that these configuration works for root certificate which is put in Windows Certstore. Currently, the chained certificate is not supported.
Certificate thumbprint is able to obtain with certutil command.
Here is an example which uses a certificate that is generated by fluent-ca-generate command:
Cert hash(sha1) is called as thumbprint in this section.
Note that -enterprise flag represents to use enterprise certstore. Please pay attention to using whether enterprise certificates store or not.
If you want to connect to a server that requires password authentication, you need to set your credentials in the configuration file:
Note that, as to the option self_hostname, you need to set the name of the server on which your out_forward instance is running. In the current implementation, it is considered invalid if your in_forward and out_forward share the same hostname.
gzip compression?Since v0.14.7, Fluentd supports transparent data compression. You can use this feature to reduce the transferred payload size.
To enable this feature, set the compress option as follows:
You do not need any configuration in the receiving server. Data compression is auto-detected and handled transparently by the destination node.
Fluentd implements an adaptive failure detection mechanism called "Phi accrual failure detector". Here is how it works:
Each in_forward node sends heartbeat packets to its out_forward server
with a regular interval.
The out_forward server records the arrival time of heartbeat packets sent
by each node.
If the server does not receive a heartbeat from one of its nodes for "a long
time", it assumes the node is down.
But how long should the server wait before detaching a node? The phi accrual failure detector answers this question by computing the probability of a node being down based on the assumption that heartbeat intervals follow normal distribution. Internally, it represents the confidence of a node being down by a continuous function φ(t) which grows as the time from the last packet increases.
For example, suppose that the historical average interval is 1 second and the standard deviation is 1, it is unlikely that the node is still being active when its heartbeat has not been received for the last 10 seconds.
For details, please read the original paper: Hayashibara, Naohiro, et al. "The φ accrual failure detector." IEEE, 2004.
Please make sure that you can communicate with port 24224 using not only TCP, but also UDP. These commands will be useful for checking the network configuration:
Please note that there is one known issue where VMware will occasionally lose small UDP packets used for heartbeat.
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is an open-source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.
