Fluentd
0.12
0.12
  • Introduction
  • Overview
    • Getting Started
    • Installation
    • Life of a Fluentd event
    • Support
    • FAQ
  • Use Cases
    • Centralized App Logging
    • Monitoring Service Logs
    • Data Analytics
    • Connecting to Data Storages
    • Stream Processing
    • Windows Event Collection
    • IoT Data Logger
  • Configuration
    • Config File Syntax
    • Routing Examples
    • Recipes
  • Deployment
    • Logging
    • Monitoring
    • Signals
    • RPC
    • High Availability Config
    • Failure Scenarios
    • Performance Tuning
    • Plugin Management
    • Trouble Shooting
    • Secure Forwarding
    • Fluentd UI
    • Command Line Option
  • Container Deployment
    • Docker Image
    • Docker Logging Driver
    • Docker Compose
    • Kubernetes
  • Input Plugins
    • tail
    • forward
    • secure_forward
    • udp
    • tcp
    • http
    • unix
    • syslog
    • exec
    • scribe
    • multiprocess
    • dummy
    • Others
  • Output Plugins
    • file
    • s3
    • kafka
    • forward
    • secure_forward
    • exec
    • exec_filter
    • copy
    • geoip
    • roundrobin
    • stdout
    • null
    • webhdfs
    • splunk
    • mongo
    • mongo_replset
    • relabel
    • rewrite_tag_filter
    • Others
  • Buffer Plugins
    • memory
    • file
  • Filter Plugins
    • record_transformer
    • grep
    • parser
    • stdout
  • Parser Plugins
    • regexp
    • apache2
    • apache_error
    • nginx
    • syslog
    • ltsv
    • csv
    • tsv
    • json
    • multiline
    • none
  • Formatter Plugins
    • out_file
    • json
    • ltsv
    • csv
    • msgpack
    • hash
    • single_value
  • Developer
    • Plugin Development
    • Community
    • Mailing List
    • Source Code
    • Bug Tracking
    • ChangeLog
    • Logo
  • Articles
    • Store Apache Logs into MongoDB
    • Apache To Riak
    • Store Apache Logs into Amazon S3
    • Before Install
    • Cep Norikra
    • Collect Glusterfs Logs
    • Common Log Formats
    • Docker Logging Efk Compose
    • Docker Logging
    • Filter Modify Apache
    • Forwarding Over Ssl
    • Free Alternative To Splunk By Fluentd
    • Data Collection to Hadoop (HDFS)
    • Data Analytics with Treasure Data
    • Install By Chef
    • Install By Deb
    • Install By Dmg
    • Install By Gem
    • Install By Rpm
    • Install From Source
    • Install On Beanstalk
    • Install On Heroku
    • Java
    • Kinesis Stream
    • Kubernetes Fluentd
    • Monitoring by Prometheus
    • Monitoring by Rest Api
    • Nodejs
    • Performance Tuning Multi Process
    • Performance Tuning Single Process
    • Perl
    • Php
    • Python
    • Quickstart
    • Raspberrypi Cloud Data Logger
    • Recipe Apache Logs To Elasticsearch
    • Recipe Apache Logs To Mongo
    • Recipe Apache Logs To S3
    • Recipe Apache Logs To Treasure Data
    • Recipe Cloudstack To Mongodb
    • Recipe Csv To Elasticsearch
    • Recipe Csv To Mongo
    • Recipe Csv To S3
    • Recipe Csv To Treasure Data
    • Recipe Http Rest Api To Elasticsearch
    • Recipe Http Rest Api To Mongo
    • Recipe Http Rest Api To S3
    • Recipe Http Rest Api To Treasure Data
    • Recipe Json To Elasticsearch
    • Recipe Json To Mongo
    • Recipe Json To S3
    • Recipe Json To Treasure Data
    • Recipe Nginx To Elasticsearch
    • Recipe Nginx To Mongo
    • Recipe Nginx To S3
    • Recipe Nginx To Treasure Data
    • Recipe Syslog To Elasticsearch
    • Recipe Syslog To Mongo
    • Recipe Syslog To S3
    • Recipe Syslog To Treasure Data
    • Recipe Tsv To Elasticsearch
    • Recipe Tsv To Mongo
    • Recipe Tsv To S3
    • Recipe Tsv To Treasure Data
    • Ruby
    • Scala
    • Splunk Like Grep And Alert Email
Powered by GitBook
On this page
  • Prerequisites
  • Setup
  • Set up a Linux server with rsyslogd and Fluentd
  • Set up nxlog on Windows
  • Test
  • Parsing JSON Logs
  • Next Step
  • Learn More

Was this helpful?

  1. Use Cases

Windows Event Collection

PreviousStream ProcessingNextIoT Data Logger

Last updated 5 years ago

Was this helpful?

In this article, we explain how to get started with collecting data from Windows machines (This setup has been tested on a 64-bit Windows 8 machine).

As of v10, Fluentd does NOT support Windows. However, there are times when you must collect data streams from Windows machines. For example:

  1. Tailing log files on Windows: collect and analyze log data from

    a Windows application.

  2. Collecting Windows Event Logs: collect event logs from your

    Windows servers for system analysis, compliance checking, etc.

If you're not familiar with Fluentd, please learn more about Fluentd first.

Prerequisites

  1. , an open source log management tool that

    runs on Windows.

  2. A Linux server (we assume Ubuntu 12 for this article)

Setup

Set up a Linux server with rsyslogd and Fluentd

  1. Get hold of a Linux server. In this example, we assume it is Ubuntu.

  2. **Make sure it has ports open for TCP. In the following example, we

    assume port 5140 is open.**

  3. . (See

    for various ways to install

    Fluentd/Treasure Agent)

  4. Edit td-agent's configuration file located at /etc/td-agent/td-agent.conf and add the following lines

     <source>
       @type tcp
       format none
       port 5140
       tag windowslog
     </source>    
     <match windowslog>
       @type stdout
     </match>
    
     The above code listens to port 5140 (UDP) and outputs the data to stdout (which is piped to `/var/log/td-agent/td-agent.log`)
  5. Start td-agent by running sudo service td-agent start

Set up nxlog on Windows

  1. nxlog onto the Windows machine you want to collect log data from.

    Open the downloaded installer and follow the instructions. By

    default, it should be installed in C:\Program Files (x86)\nxlog

  2. Create an nxlog config file as follows and save it as nxlog.conf:

     #define ROOT C:\Program Files\nxlog
     define ROOT C:\Program Files (x86)\nxlog
    
     Moduledir %ROOT%\modules
     CacheDir %ROOT%\data
     Pidfile %ROOT%\data\nxlog.pid
     SpoolDir %ROOT%\data
     LogFile %ROOT%\data\nxlog.log
    
     <Input in>
       Module im_file
       File 'C:\Users\SomeUser\Desktop\nxlog_test.log' #Put the file to be tailed here.
       SavePos TRUE
       InputType LineBased
     </Input>
    
     <Output out>
       Module om_tcp
       Host LINUX_MACHINE_RUNNING_FLUENTD
       Port 5140
     </Output> 
    
     <Route r>
       Path in => out
     </Route>

    This configuration will send each line of the log file (see the File parameter inside \<Input in>...\</Input>) as a syslog message to a remote Fluentd/Treasure Agent instance.

Test

  1. Go to nxlog's directory (in Powershell or Command Prompt) and run the following command:

     \nxlog.exe -f -c  <path to nxlog.conf>

    The "-f" option runs nxlog in the foreground (this is for testing). If this is for production, you would want to turn it into a Windows Service.

  2. Once nxlog is running, add a new line "Windows is awesome" into the tailed file like this:

     echo Windows is awesome >> 'C:\Users\SomeUser\Desktop\nxlog_test.log'
  3. Now, go to the Linux server and run

     $ sudo tail -f /var/log/td-agent/td-agent.log
     ...
     ...
     ...
     2014-12-20 02:19:36 +0000 windowslog: {"message":"Windows is awesome \r"}
  4. You successfully sent data from a Windows machine to a remote Fluentd instance running on Linux.

Parsing JSON Logs

If you are sending JSON logs on Windows to Fluentd, Fluentd can parse them as they come in. To do, simply change Fluentd's configuration as follows. Note the change from format none to format json. (See this article for more details about the parser plugins)

<source>
  @type tcp
  format json
  port 5140
  tag windowslog
</source>    
<match windowslog>
  @type stdout
</match>

Then, if you add a new line to the file on your windows machine like this:

echo {"name":"Sadayuki", "age":27} >> 'C:\Users\SomeUser\Desktop\nxlog_test.log'

On the Linux machine running Fluentd, you see the following line:

2014-12-20 02:22:44 +0000 windowslog: {"name":"Sadayuki","age":27}

Next Step

This example showed that we can collect data from a Windows machine and send it to a remote Fluentd instance. However, the data is not terribly useful because each line of data is placed into the "message" field as unstructured text. For production purposes, you would probably want to write a plugin/extend the syslog plugin so that you can parse the "message" field in the event.

Learn More

  • Fluentd Get Started

Follow and download a copy of

If this article is incorrect or outdated, or omits critical information, please . is a open source project under . All components are available under the Apache 2 License.

What is Fluentd?
nxlog
Install td-agent
here
this link
Fluentd Architecture
let us know
Fluentd
Cloud Native Computing Foundation (CNCF)