Support Questions
Find answers, ask questions, and share your expertise

Need instructions to start data ingest on metron

Contributor

Hello, I managed to install single node metron via vagrant and I reached a point where Kibana status is Green & All hadoop services all in good state. But I am clueless of how to start ingesting pcap, netflow or bro.

Looking at the github codebase of 0.2.BETA, it looks like you guys so far only implemented data ingestion sensors for pcap using fastcapa and bro using bro-plugin-kafka. Correct me if I am wrong about this. And please provide some instructions for the following:

1) How to start data ingestion: pcap or yaf netflow ?

2) Check status of ingestion

3) Check storm processing

4) Kibana UI: where to look at the alerts ?

Engineer
1 REPLY 1

Rising Star

(1) Bro, Snort and YAF ingestion are started by default during the deployment process. You should eventually start to see data hit your Kibana dashboard auto-magically. This may take a few minutes just because of the resource constraints of running on a single node.

One easy way to start and stop all the components for ingestion of Bro, Snort, and YAF is to use the Monit web interface. Login to http://localhost:2812 using admin/monit. More information is in this README. If you intend to work with single node, I would suggest that you only ingest one data source at a time.

(2) Ingestion involves a few components; the sensors, Kafka, Storm, and Elasticsearch. To check the status of ingestion you would need to use the tools provided by each of these components.

I know this is not ideal, but we might have a management user interface in the works that should make this task much simpler.

(3) A good place to start is to use the Storm UI. You can find a link to the Storm UI by logging into Ambari http://localhost:8080 as admin/admin. Click on the "Storm" service and then there should be a "Quick Links" (or something named similarly) that will provide a link to the Storm UI.

(4) We don't have a widget in the default Kibana dashboard dedicated to alerts. But the beauty of Kibana is that you can easily create your own. Just create a search in Kibana that looks for the 'is_alert' field set to 'true'. You can then visualize that however you think is best for your purposes.