Support Questions
Find answers, ask questions, and share your expertise

How to collect windows event log using Nifi

Highlighted

How to collect windows event log using Nifi

Explorer

Hi, 

 

I have a domain environment including windows servers. In order to collect all the lof i have implemented WEF and all logs are been pushed to a collector.

Now, I wish to inject the logs from the collector to a NIFI server - What is the best way to do it? what kind of listener i should use? Pull or Push?

 

Thanks

5 REPLIES 5
Highlighted

Re: How to collect windows event log using Nifi

Cloudera Employee

@dzbeda You can use one of below approach.

1. Install minifi on windows machine and send logs from minifi to NiFi [Push] 

2. You can use ConsumeWindowsEventLog to get event logs in NiFi [Pull]

Highlighted

Re: How to collect windows event log using Nifi

Master Guru

@dzbeda 

 

Just to add to this, MiNiFi offers a C++ agent.  There are many users out there using MiNiFi CPP collecting Wiindows event logs and forwarding them to nifi via invokehttp (on MiNiFi CPP) to listenhttp (on NiFi).

Thanks,

Matt

Highlighted

Re: How to collect windows event log using Nifi

@dzbeda In a previous lifetime I accomplished getting windows log data and windows metrics using Elastic Beats.  There is one winlogbeat which is great.  Even using regular file beats you can make custom listener.  This leverages the ELK stack, (elasticsearch, logstash, kibana, beats), but is an interesting look, and connecting in NiFi through the elk indexes on that log data.

 

The other method i have used is Minifi, as suggested to @ashinde, but this is a technical challenge with some difficult hurdles to get a data flow working in windows and wired up to Nifi.   If you take this route I would challenge you to create an article here in the community to share your solution.

 

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.

 

Thanks,

Steven

Highlighted

Re: How to collect windows event log using Nifi

Rising Star

Did you know that Nifi was developed in the Java language?

Pure Java cannot handle Windows event logs.
The method using JNA is recommended. Please refer to the following link.

 

https://code.dblock.org/2010/09/30/jna-reading-windows-event-log-entries-in-java.html

Re: How to collect windows event log using Nifi

Master Guru

MiNiFi offers CPP version that is well suited for Windows event log ingestion.