@Simon Elliston Ball Wonderful news! Thank you for the snapshot and the information, I was able to run the script, but I need to work on the query syntax. I assume the query syntax is in Stellar. ... View more

I was able to upgrade ambari to 2.6.1 successfully. Thank you again for your help @Jonathan Hurley ... View more

Thank you very much @Jasper I was able to do that for the indexing topology, but how do you set that for parsing and enrichment topology? I still see a large number of failed under indexing topology but nothing obvious in logs. Occasionally, I see kafka coordinate mark and discover dead topic and I don't know how to fix that, but it goes away after awhile ... View more

BTW, I realized the file global.json gets overwritten by metron/config on Ambari. There's a section called global.json template Just in case someone might find it useful. ... View more

Thank you @rmerriman for your response. I tried the replay route; however tcpreplay doesn't preserve the timestamp of the original packets and I have yet to find a workaround for that. I'm going to try with the second option that you suggested. I'll look into data migration between ES/HDFS clusters. Thank you! ... View more

Thank you all feedbacks. Simon, Thank you for pointing out many useful things. It would be helpful if on Metron Document, we makr what's deprecated and what's no longer supported. I would like to understand why Metron doesn't support bringing in PCAP data to add to Metron Cluster. I thought the whole idea is to have the data in one place. My work around would be to use tshark to extract PCAP metadata and push it to Metron manually. What's your take on that? I'll spend more time with query and inspector tools for PCAP, however, we like to visualize our data on a dashboard along with our other network traffic data collected. I do use Ambari to manage services, however it doesn't have status on the sensors like yaf, bro, and snort like Monit does. Unless, I'm missing a configuration/installation steps to add sensor services to Ambari. Thank you for your time and feedbacks. ... View more

No problem. Please reach out if there is anything else we can help you with ... View more

Thank you @jsirota for the explaination. I think I got the first part comfortably. However, the second part is still fuzzy to me where we narrow down to certain data to export out to PCAP format in order to view them in wireshark. I was looking up for Metron meetup around NOVA/MD area, but couldn't find any. There are so much with Metron I would like to learn and understand better. I started to tap into our company network interface instead of the tap0 switch we created and I started to run into more issues with services being down. ... View more

> The pcap data stored in HDFS is sequence files. How do you view them in Wireshark? My guess would be somehow get the pcap_inspector service to spit out the result of the filter in PCAP format? @Arian Trayen As @cstella mentioned, "pcap_query" does exactly that. It will output a libpcap-compliant file that you can open with Wireshark. ... View more