Windows Event Collection
In this article, we explain how to get started with collecting data from Windows machines (This setup has been tested on a 64-bit Windows 8 machine).
As of v10, Fluentd does NOT support Windows. However, there are times when you must collect data streams from Windows machines. For example:
- 1.Tailing log files on Windows: collect and analyze log data froma Windows application.
- 2.Collecting Windows Event Logs: collect event logs from yourWindows servers for system analysis, compliance checking, etc.
If you're not familiar with Fluentd, please learn more about Fluentd first.
- 1.runs on Windows.
- 2.A Linux server (we assume Ubuntu 12 for this article)
- 1.Get hold of a Linux server. In this example, we assume it is Ubuntu.
- 2.**Make sure it has ports open for TCP. In the following example, weassume port 5140 is open.**
- 3.
- 4.Edit td-agent's configuration file located at
/etc/td-agent/td-agent.conf
and add the following lines<source>@type tcpformat noneport 5140tag windowslog</source><match windowslog>@type stdout</match>The above code listens to port 5140 (UDP) and outputs the data to stdout (which is piped to `/var/log/td-agent/td-agent.log`) - 5.Start td-agent by running
sudo service td-agent start
- 1.nxlog onto the Windows machine you want to collect log data from.Open the downloaded installer and follow the instructions. Bydefault, it should be installed in
C:\Program Files (x86)\nxlog
- 2.Create an nxlog config file as follows and save it as
nxlog.conf
:#define ROOT C:\Program Files\nxlogdefine ROOT C:\Program Files (x86)\nxlogModuledir %ROOT%\modulesCacheDir %ROOT%\dataPidfile %ROOT%\data\nxlog.pidSpoolDir %ROOT%\dataLogFile %ROOT%\data\nxlog.log<Input in>Module im_fileFile 'C:\Users\SomeUser\Desktop\nxlog_test.log' #Put the file to be tailed here.SavePos TRUEInputType LineBased</Input><Output out>Module om_tcpHost LINUX_MACHINE_RUNNING_FLUENTDPort 5140</Output><Route r>Path in => out</Route>This configuration will send each line of the log file (see the File parameter inside \<Input in>...\</Input>) as a syslog message to a remote Fluentd/Treasure Agent instance.
- 1.Go to nxlog's directory (in Powershell or Command Prompt) and run the following command:\nxlog.exe -f -c <path to nxlog.conf>The "-f" option runs nxlog in the foreground (this is for testing). If this is for production, you would want to turn it into a Windows Service.
- 2.Once nxlog is running, add a new line "Windows is awesome" into the tailed file like this:echo Windows is awesome >> 'C:\Users\SomeUser\Desktop\nxlog_test.log'
- 3.Now, go to the Linux server and run$ sudo tail -f /var/log/td-agent/td-agent.log.........2014-12-20 02:19:36 +0000 windowslog: {"message":"Windows is awesome \r"}
- 4.You successfully sent data from a Windows machine to a remote Fluentd instance running on Linux.
If you are sending JSON logs on Windows to Fluentd, Fluentd can parse them as they come in. To do, simply change Fluentd's configuration as follows. Note the change from
format none
to format json
. (See this article for more details about the parser plugins)<source>
@type tcp
format json
port 5140
tag windowslog
</source>
<match windowslog>
@type stdout
</match>
Then, if you add a new line to the file on your windows machine like this:
echo {"name":"Sadayuki", "age":27} >> 'C:\Users\SomeUser\Desktop\nxlog_test.log'
On the Linux machine running Fluentd, you see the following line:
2014-12-20 02:22:44 +0000 windowslog: {"name":"Sadayuki","age":27}
This example showed that we can collect data from a Windows machine and send it to a remote Fluentd instance. However, the data is not terribly useful because each line of data is placed into the "message" field as unstructured text. For production purposes, you would probably want to write a plugin/extend the syslog plugin so that you can parse the "message" field in the event.
If this article is incorrect or outdated, or omits critical information, please let us know. Fluentd is a open source project under Cloud Native Computing Foundation (CNCF). All components are available under the Apache 2 License.
Last modified 4yr ago