How to collect windows event log using Nifi

[#1]

Hi, 

I have a domain environment including windows servers. In order to collect all the lof i have implemented WEF and all logs are been pushed to a collector.

Now, I wish to inject the logs from the collector to a NIFI server - What is the best way to do it? what kind of listener i should use? Pull or Push?

Thanks

[#2]

@dzbeda You can use one of below approach.

1. Install minifi on windows machine and send logs from minifi to NiFi [Push] 

2. You can use ConsumeWindowsEventLog to get event logs in NiFi [Pull]

[#3]

@dzbeda 

Just to add to this, MiNiFi offers a C++ agent.  There are many users out there using MiNiFi CPP collecting Wiindows event logs and forwarding them to nifi via invokehttp (on MiNiFi CPP) to listenhttp (on NiFi).

Thanks,

Matt

[#4]

Hello,

I'm a learner & i would like to use the method you made mentioned here to collect logs in a remote server & send to Nifi. Please, can you put me through because i have been battling with how to build a msi before the real implementation.

Thank you so much.

[#5]

@dzbeda In a previous lifetime I accomplished getting windows log data and windows metrics using Elastic Beats.  There is one winlogbeat which is great.  Even using regular file beats you can make custom listener.  This leverages the ELK stack, (elasticsearch, logstash, kibana, beats), but is an interesting look, and connecting in NiFi through the elk indexes on that log data.

The other method i have used is Minifi, as suggested to @ashinde, but this is a technical challenge with some difficult hurdles to get a data flow working in windows and wired up to Nifi.   If you take this route I would challenge you to create an article here in the community to share your solution.

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.

Thanks,

Steven

[#6]

Did you know that Nifi was developed in the Java language?

Pure Java cannot handle Windows event logs.
The method using JNA is recommended. Please refer to the following link.

https://code.dblock.org/2010/09/30/jna-reading-windows-event-log-entries-in-java.html

[#7]

MiNiFi offers CPP version that is well suited for Windows event log ingestion.