Created on 12-31-2018 08:35 AM - edited 09-16-2022 07:01 AM
Hi
I want to send Windows event log to HCP ( with any agent like winlogbeats or etc ) but I don't know how to do this ? can you provide solution ?
Thanks
Created 12-31-2018 08:44 AM
Hi @haco fayik,
as a starting point you need to push data into a parser specific Kafka topic (you can call the topic "windows-event-log"), and configure a parser in the Metron Management UI and start it. In the parser configuration you configure Metron, from which Kafka topic the messages are picked up ("windows-event-log" in our case) and how to parse the incoming messages.
NiFi is a great tool to collect data from various sources and push it into Kafka.
Maybe my article helps you: https://datahovel.com/2018/07/18/how-to-onboard-a-new-data-source-in-apache-metron/
If you have more specific questions, don't hesitate to ask!
Created 12-31-2018 08:44 AM
Hi @haco fayik,
as a starting point you need to push data into a parser specific Kafka topic (you can call the topic "windows-event-log"), and configure a parser in the Metron Management UI and start it. In the parser configuration you configure Metron, from which Kafka topic the messages are picked up ("windows-event-log" in our case) and how to parse the incoming messages.
NiFi is a great tool to collect data from various sources and push it into Kafka.
Maybe my article helps you: https://datahovel.com/2018/07/18/how-to-onboard-a-new-data-source-in-apache-metron/
If you have more specific questions, don't hesitate to ask!
Created 12-31-2018 12:33 PM
hi @Stefan Kupstaitis-Dunkler,
Thank you so much for your answer ,
if I have 5 windows server and workstation , I should install nifi on each host or I can use one nifi server for all hosts ?
How to send data ( event log) to nifi ?
Created 12-31-2018 12:59 PM
There's many ways to do this. You should probably search this community in the NiFi section or get familiar with NiFi in general.
However, as a a short overview, the most common cases for Metron ingestion, I'm encountering in the field are:
Note: If your Kafka cluster is secured with Kerberos, this might influence your choice.
Created 01-01-2019 11:33 AM
thank you very much @Stefan Kupstaitis-Dunkler
Created on 01-07-2019 12:24 PM - edited 08-17-2019 03:17 PM
I Installed winlogbeats on Windows workstation with below config :
output.logstash: hosts: ["nifi.node.srv:5098"]
and I use this nifi processors to stream event to metron
listenbeats config :
Publishkafka cofig :
Nifi Data provenance in publishkafka processor :
and I create sensor in Management UI with logstash parser and winlogtop topic ( kafka) . now I can't see any log data in alert UI . what's problem ?
Thanks
Created 01-07-2019 12:27 PM
Hi @haco fayik
That looks great. Sounds like you got around the initial problem of ingesting data into Metron.
There could be multiple reasons, e.g. parser, enrichment and indexing topologies not running or being misconfigured.
Would you create a new question for this and provide more details, such as worker logs of those topologies?
Would you also mark the answer that helped you most solve the ingest problem as "Best Answer"?
thanks!
Created 01-08-2019 08:43 AM
Thanks @Stefan Kupstaitis-Dunkler,
I marked best answer and I will create a new question for this problem . Can you provide location of these log file?
I confused that Can I use metron for Collect windows and linux hosts and network devices log for security purpose ? ( Threat detection and etc)
Please accept my thanks for your helps