Archives of Support Questions (Read Only)

nnat25191 · ‎10-03-2017

I've installed and configured pcap service and replay for our apache metron. The pcap data has been captured and stored in HDFS /apps/metron/pcap. However, I could not find information of how to index these files into ElasticSearch and make it available on Apache Metron dashboard. I found a related thread, but didn't answer my question or I'm just confused? https://community.hortonworks.com/questions/36622/how-to-use-the-metron-ui-to-see-the-pcap-data.html

1. How to get the pcap data collected/stored in HDFS indexed into ElasticSearch?

2. How to get the pcap panel on Metron dashboard like the old version of Metron?

Any feedback is greatly appreciated.

nallen · ‎10-04-2017

> 1. How to get the pcap data collected/stored in HDFS indexed into ElasticSearch?

In Apache Metron there is not a mechanism to ingest raw pcap data into Elasticsearch. I have found a search index like Elasticsearch more useful for higher level meta information like flows.

There is a tool called Pcap Query to search and retrieve slices of the raw pcap stored in HDFS. This queries against the data stored in HDFS and returns a libpcap-compliant file containing the raw pcap data that you can then load into 3rd party tools like Wireshark.

> 2. How to get the pcap panel on Metron dashboard like the old version of Metron?

The Pcap Panel from the original OpenSOC project was not carried forward due to technical limitations.

View solution in original post

nnat25191 · ‎11-15-2017

Thank you @jsirota for the explaination. I think I got the first part comfortably. However, the second part is still fuzzy to me where we narrow down to certain data to export out to PCAP format in order to view them in wireshark.

I was looking up for Metron meetup around NOVA/MD area, but couldn't find any. There are so much with Metron I would like to learn and understand better. I started to tap into our company network interface instead of the tap0 switch we created and I started to run into more issues with services being down.

Cloudera Community

Archives of Support Questions (Read Only)

Pcap data index to ElasticSearch