The out_http Output plugin writes records into via HTTP/HTTPS.

This plugin is introduced since fluentd v1.7.0.

Example Configuration

out_http is included in Fluentd's core. No additional installation process is required.

<match pattern>
  @type http
​
  endpoint http://logserver.com:9000/api
  open_timeout 2
​
  <format>
    @type json
  </format>
  <buffer>
    flush_interval 10s
  </buffer>
</match>

Please see the Config File article for the basic structure and syntax of the configuration file. For <buffer> section, please check Buffer section cofiguration.

Parameters

@type

The value must be http.

endpoint

The endpoint for HTTP request. If you want to use HTTPS, use https prefix.

# Use HTTP
endpoint http://example.com/api
# USe HTTPS. You can set additional HTTPS parameters like tls_xxx
endpoint https://example.com/api

The endpoint parameter supports placeholders, so you can embed time, tag and record fields in the endpoint. Here is an example:

endpoint http://example.com/api/${tag}-${key}
<buffer tag,key>
  # buffer parameters
</buffer>

See Buffer section configurations for more details.

http_method

The method for HTTP request

proxy

The proxy for HTTP request

content_type

Content-Type for HTTP request. out_http automatically set Content-Type for built-in formatters when this parameter is not specified. Here is a table:

• json(json_array: false): application/x-ndjson

• json(json_array: true): application/json

• csv: text/csv

• tsv, ltsv: text/tab-separated-values

• msgpack: application/x-msgpack

• out_file, single_value, stdout, hash: text/plain

json_array

Using array format of JSON. This parameter is used and valid only for json format. When json_array as true, Content-Type should be application/json and be able to use JSON data for the HTTP request body.

<format> directive

The format of the payload. The default @type is json. Here is single_value example:

<format>
  @type single_value
</format>

See formatter article for more detail.

headers

Additional headers for HTTP request

headers {"key1":"value1", "key2":"value2"}

open_timeout

The connection open timeout in seconds. See also Ruby document​

read_timeout

The read timeout in seconds. See also Ruby document​

ssl_timeout

The TLS timeout in seconds

tls_ca_cert_path

The CA certificate path for TLS

tls_client_cert_path

The client certificate path for TLS

tls_private_key_path

The client private key path for TLS

tls_private_key_passphrase

The client private key passphrase for TLS

tls_verify_mode

The verify mode of TLS

tls_version

The default version of TLS

tls_ciphers

The cipher suites configuration of TLS.

error_response_as_unrecoverable

Raise UnrecoverableError when the response code is non success, 1xx/3xx/4xx/5xx. If false, out_http logs error message instead of raising UnrecoverableError.

See also Handling Unrecoverable Errors​

retryable_response_codes

The list of retryable response code. If the response code is included in this list, out_http retries the buffer flush.

<auth> section

Specify HTTP authentication.

<auth>
  method basic
  username fluentd
  password hello
</auth>

method

The method for HTTP authentication. Now only basic.

username

The username for basic authentication

password

The password for basic authentication

Common Output / Buffer parameters

For common output / buffer parameters, please check the following articles.

• ​Output Plugin Overview​

• ​Buffer Section Configuration​

The payload content

out_http's request body depends on <format> configuration. For example, the default setting generates newline delimited json like below:

# \n is added by `add_newline true` parameter in <format>
{"key1":"value1","key2":"value2",...}\n
{"key1":"value1","key2":"value2",...}\n
{"key1":"value1","key2":"value2",...}\n
{"key1":"value1","key2":"value2",...}\n
...

With @type single_value:

log line1\n
log line2\n
log line3\n
log line4\n
...

Troubleshooting

400 Bad request between out_http and in_http

When getting the following error:

#0 got unrecoverable error in primary and no secondary error_class=Fluent::UnrecoverableError error="400 Bad Request 400 Bad Request\n'json' or 'msgpack' parameter is required\n"
#0 bad chunk is moved to /tmp/fluent/backup/worker0/object_3ff8a73edae8/5a71a08ca19b1b343c8dce1b74c9a963.log

Users should be specify json format with json_array as true for out_http configuration:

<match **>
  @type http
  endpoint http://some.your.http.endpoint:9811/your-awesome-path
  <format>
    @type json
  </format>
  json_array true
  <buffer>
    flush_interval 2s
  </buffer>
</match>

And receiver in_http configuration should be:

<source>
  @type http
  port 9811
  bind 0.0.0.0
  <parse>
    @type json
  </parse>
</source>

Or specify msgpack format:

<match **>
  @type http
  endpoint http://some.your.http.endpoint:9882/your-awesome-path
  <format>
    @type msgpack
  </format>
  <buffer>
    flush_interval 2s
  </buffer>
</match>

And receiver in_http configuration should be:

<source>
  @type http
  port 9882
  bind 0.0.0.0
  <parse>
    @type msgpack
  </parse>
  <format>
    @type json
  </format>
</source>

But, we recommend to use in/out forward plugin to communicate with two Fluentds due to at-most-once and at-least-once semantics for rigidty.

If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.