Comment on page
Install by RPM Package (Red Hat Linux)
This article explains how to install stable versions of
fluent-package
rpm packages, the stable Fluentd distribution packages maintained by Fluentd Project and calyptia-fluentd
which is maintained by Calyptia, Inc..Fluentd is written in Ruby for flexibility, with performance-sensitive parts in C. However, some users may have difficulty installing and operating a Ruby daemon.
That is why Fluentd Project provides the stable distribution of Fluentd, called
fluent-package
(formerly known as td-agent
). The differences between Fluentd and fluent-package
can be found here.Our Calyptia also knows that Fluentd is written in Ruby for flexibility, with performance-sensitive parts in C. However, some users may have difficulty installing and operating a Ruby daemon. And
td-agent
is still seated on Ruby 2.7 due to compatibility reasons and Ruby versioning policy, calyptia-fluentd
uses Ruby 3 instead of Ruby 2.7 for now.That is why Calyptia, Inc. provides the alternative stable distribution of Fluentd, called
calyptia-fluentd
. The differences between td-agent
and calyptia-fluentd
are bundled and running Ruby versions for now.This installation guide is for
fluent-package
v5 and calyptia-fluentd
v1. fluent-package
v5 and calyptia-fluentd
use fluentd v1 in the core. See fluent-package-v5-vs-td-agent or td-agent-v2-vs-v3-vs-v4 for the comparison and supported OS.NOTE:
- About deprecated Treasure Agent (td-agent) 3 will not be maintained anymore, see Install by DEB Package v3.
fluent-package
will be shipped in two flavors - normal release version and LTS (Long Term Support) version. See Scheduled support lifecycle announcement about Fluent Package about difference between this two flavors.
It is highly recommended to set up
ntpd
on the node to prevent invalid timestamps in the logs. See Pre-installation Guide.Download and execute the install script with
curl
:# fluent-package 5 (LTS)
$ curl -fsSL https://toolbelt.treasuredata.com/sh/install-redhat-fluent-package5-lts.sh | sh
# fluent-package 5
$ curl -fsSL https://toolbelt.treasuredata.com/sh/install-redhat-fluent-package5.sh | sh
Executing this script will automatically install
fluent-package
on your machine. This shell script registers a new rpm
repository at /etc/yum.repos.d/fluent-package.repo
(or /etc/yum.repos.d/fluent-package-lts.repo
) and installs fluent-package
.We use
$releasever
for repository path in the script and $releasever
should be the major version only like "7"
. If your environment uses some other format like "7.2"
, change it to the major version only or set up .repo file manually.For Amazon Linux 2023:
# fluent-package 5 (LTS)
$ curl -fsSL https://toolbelt.treasuredata.com/sh/install-amazon2023-fluent-package5-lts.sh | sh
# fluent-package 5
$ curl -fsSL https://toolbelt.treasuredata.com/sh/install-amazon2023-fluent-package5.sh | sh
For Amazon Linux 2:
# fluent-package 5 (LTS)
$ curl -fsSL https://toolbelt.treasuredata.com/sh/install-amazon2-fluent-package5-lts.sh | sh
# fluent-package 5
$ curl -fsSL https://toolbelt.treasuredata.com/sh/install-amazon2-fluent-package5.sh | sh
Use
/usr/lib/systemd/system/fluentd
service to start
, stop
, or restart
the agent:$ sudo systemctl start fluentd.service
$ sudo systemctl status fluentd.service
* fluentd.service - fluentd: All in one package of Fluentd
Loaded: loaded (/usr/lib/systemd/system/fluentd.service; enabled; vendor preset: disabled)
Active: active (running) since Wed 2023-08-02 05:35:16 UTC; 41s ago
Docs: https://docs.fluentd.org/
Process: 1901 ExecStart=/opt/fluent/bin/fluentd --log $FLUENT_PACKAGE_LOG_FILE --daemon /var/run/fluent/fluentd.pid $FLUENT_PACKAGE_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 1907 (fluentd)
CGroup: /system.slice/fluentd.service
|-1907 /opt/fluent/bin/ruby /opt/fluent/bin/fluentd --log /var/log/fluent/fluentd.log --daemon /var/run/fluent/fluentd.pid
`-1910 /opt/fluent/bin/ruby -Eascii-8bit:ascii-8bit /opt/fluent/bin/fluentd --log /var/log/fluent/fluentd.log --daemon /var/ru...
To customize
systemd
behavior, put your fluentd.service
in /etc/systemd/system
.NOTE: In
fluent-package
v5, the path is different i.e. /opt/fluent/bin
instead of /opt/td-agent/bin
.The default configuration (
/etc/fluent/fluentd.conf
) is to receive logs at an HTTP endpoint and route them to stdout
. For fluentd
logs, see /var/log/fluent/fluentd.log
.You can post sample log records with
curl
command:$ curl -X POST -d 'json={"json":"message"}' http://localhost:8888/debug.test
$ tail -n 1 /var/log/fluent/fluentd.log
2023-08-02 05:37:29.185634777 +0000 debug.test: {"json":"message"}
It is highly recommended to set up
ntpd
on the node to prevent invalid timestamps in the logs. See Pre-installation Guide.Download and execute the install script with
curl
:# calyptia-fluentd 1
$ curl -L https://calyptia-fluentd.s3.us-east-2.amazonaws.com/calyptia-fluentd-1-redhat.sh | sh
Executing this script will automatically install
calyptia-fluentd
on your machine. This shell script registers a new rpm
repository at /etc/yum.repos.d/Calyptia-Fluentd.repo
and installs calyptia-fluentd
.We use
$releasever
for repository path in the script and $releasever
should be the major version only like "7"
. If your environment uses some other format like "7.2"
, change it to the major version only or set up TD repository manually.For CentOS Stream 8:
# calyptia-fluentd 1
$ curl -L https://calyptia-fluentd.s3.us-east-2.amazonaws.com/calyptia-fluentd-1-centos-stream.sh
For Amazon Linux 2:
# calyptia-fluentd 1
$ curl -L https://calyptia-fluentd.s3.us-east-2.amazonaws.com/calyptia-fluentd-1-amazon-2.sh | sh
calyptia-fluentd
only provides systemd's unit file:Use
/usr/lib/systemd/system/calyptia-fluentd
script to start
, stop
, or restart
the agent:$ sudo systemctl start calyptia-fluentd.service
$ sudo systemctl status calyptia-fluentd.service
● calyptia-fluentd.service - calyptia-fluentd: Fluentd based data collector for Calyptia Services
Loaded: loaded (/usr/lib/systemd/system/calyptia-fluentd.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2021-05-31 01:37:47 UTC; 4h 38min ago
Docs: https://docs.fluentd.org/
Process: 694 ExecStart=/opt/calyptia-fluentd/bin/fluentd --log $CALYPTIA_FLUENTD_LOG_FILE --daemon /var/run/calyptia-fluentd/calyptia-fluentd.pid $CALYPTIA_FLUENTD_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 1365 (fluentd)
CGroup: /system.slice/calyptia-fluentd.service
├─1365 /opt/calyptia-fluentd/bin/ruby /opt/calyptia-fluentd/bin/fluentd --log /var/log/calyptia-fluentd/calyptia-fluentd.log --daem...
└─1368 /opt/calyptia-fluentd/bin/ruby -Eascii-8bit:ascii-8bit /opt/calyptia-fluentd/bin/fluentd --log /var/log/calyptia-fluentd/cal...
To customize
systemd
behavior, put your calyptia-fluentd.service
in /etc/systemd/system
.Please make sure your configuration file path is:
/etc/calyptia-fluentd/calyptia-fluentd.conf
The default configuration (
/etc/calyptia-fluentd/calyptia-faluentd.conf
) is to receive logs at an HTTP endpoint and route them to stdout
. For calyptia-fluentd
logs, see /var/log/calyptia-fluentd/calyptia-fluentd.log
.You can post sample log records with
curl
command:$ curl -X POST -d 'json={"json":"message"}' http://localhost:8888/debug.test
$ sudo tail -n 1 /var/log/calyptia-fluentd/calyptia-fluentd.log
2021-05-31 06:19:04.415878392 +0000 debug.test: {"json":"message"}
You are now ready to collect real logs with Fluentd. Refer to the following tutorials on how to collect data from various sources:
- Basic Configuration
For further steps, follow these:
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is an open-source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.
Last modified 3mo ago