Support Questions
Find answers, ask questions, and share your expertise
Alert: Please see the Cloudera blog for information on the Cloudera Response to CVE-2021-4428

No data in HDFS

Expert Contributor

I've followed this instruction to install Metron through vagrant but getting no data in hdfs when running the command below. I do see data for Bro, YAF, Alerts in Metron UI. No Snort and PCAP data though. The path /apps/metron/patterns do have some data.

$ hadoop fs -ls /apps/metron/enrichment/indexed
Found 2 items
drwxrwxr-x   - storm hadoop          0 2016-05-03 19:44 /apps/metron/enrichment/indexed/bro_doc
drwxrwxr-x   - storm hadoop          0 2016-05-03 19:45 /apps/metron/enrichment/indexed/yaf_doc

Rising Star

In TP1, the canned pcap data that we are feeding Metron does not trigger any of the Snort alerts. If you work with 'master', we added a "sensor test mode" which is on by default in the Vagrant deployment. This specifically configures a Snort rule to fire on every packet that it sees.

Also in TP1, the pcap data was not designed to land in HDFS, but instead Hbase. Since then, we have added a great new means of capturing pcap data that does store the data in HDFS.

If you fire up 'Single Node Vagrant' from 'master', open your browser and go to http://node1:2812 then login as admin/monit. From there you should see a screen listing all of the installed sensors and topologies. They will all be off, if I am not mistaken. Click on 'pcap-topology' then 'Start Service'. Then click on 'pcap' and then 'Start Service'. This will fire up the pcap ingestion where you should see packet data landing in HDFS.

Let me know if that works for you. Thanks!

Rising Star

I just remembered that the Monit stuff (aka http://node1:2812) is not in master yet. Its is in a PR 103. You can try what I mentioned after that gets merged in.

Prior to that you can SSH to the host and execute the equivalent scripts at /usr/metron/<version>/bin. That would get the job done too.