Fluentd

Java

The fluent-logger-java library is used to post records from Java applications to Fluentd.

This article explains how to use it.

Prerequisites

• Basic knowledge of Java
• Basic knowledge of Fluentd
• Java 7 or higher

Installing Fluentd

Please refer to the following document to install Fluentd:

• Installation

Modifying the Config File

Configure Fluentd to use the forward input plugin as its data source:

<source>
  @type forward
  port 24224
</source>
<match fluentd.test.**>
  @type stdout
</match>

Restart Fluentd after configuring.

# for fluent-package (rpm/deb)
$ sudo systemctl restart fluentd

Using fluent-logger-java

Add the following dependency configuration to pom.xml. The logger's revision information can be found in CHANGES.txt:

<dependencies>
  ...
  <dependency>
    <groupId>org.fluentd</groupId>
    <artifactId>fluent-logger</artifactId>
    <version>${logger.version}</version>
  </dependency>
  ...
</dependencies>

Here's a sample Java test application:

import java.util.HashMap;
import java.util.Map;
import org.fluentd.logger.FluentLogger;

public class Main {
    private static FluentLogger LOG = FluentLogger.getLogger("fluentd.test");

    public void doApplicationLogic() {
        // ...
        Map<String, Object> data = new HashMap<String, Object>();
        data.put("from", "userA");
        data.put("to", "userB");
        LOG.log("follow", data);
        // ...
    }
}

More information on the Java API can be found here.

Executing the above program will send the logs to Fluentd:

$ java -jar test.jar

The logs should be output to /var/log/fluent/fluentd.log or the standard output of the Fluentd process via stdout output plugin.

Production Deployments

Output Plugins

Various output plugins are available for writing records to other destinations:

• Examples
  • Store Apache Logs into Amazon S3
  • Store Apache Logs into MongoDB
  • Data Collection into HDFS
• List of Plugin References
  • Output to Another Fluentd
  • Output to MongoDB or MongoDB ReplicaSet
  • Output to Hadoop
  • Output to File
  • etc...

High-Availability Configurations of Fluentd

For high-traffic websites (more than 5 application nodes), we recommend using the high-availability configuration for Fluentd. This will improve the reliability of data transfer and query performance.

• High-Availability Configurations of Fluentd

Monitoring

Monitoring Fluentd itself is also important. The article below describes the general monitoring methods for Fluentd:

• Monitoring Fluentd