Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

No data in HDFS

Highlighted

No data in HDFS

Expert Contributor

I've followed this instruction to install Metron through vagrant https://community.hortonworks.com/articles/24818/metron-tech-preview-1-install-instructions-on-sing.... but getting no data in hdfs when running the command below. I do see data for Bro, YAF, Alerts in Metron UI. No Snort and PCAP data though. The path /apps/metron/patterns do have some data.

$ hadoop fs -ls /apps/metron/enrichment/indexed
Found 2 items
drwxrwxr-x   - storm hadoop          0 2016-05-03 19:44 /apps/metron/enrichment/indexed/bro_doc
drwxrwxr-x   - storm hadoop          0 2016-05-03 19:45 /apps/metron/enrichment/indexed/yaf_doc
2 REPLIES 2

Re: No data in HDFS

Rising Star

In TP1, the canned pcap data that we are feeding Metron does not trigger any of the Snort alerts. If you work with 'master', we added a "sensor test mode" which is on by default in the Vagrant deployment. This specifically configures a Snort rule to fire on every packet that it sees.

Also in TP1, the pcap data was not designed to land in HDFS, but instead Hbase. Since then, we have added a great new means of capturing pcap data that does store the data in HDFS.

If you fire up 'Single Node Vagrant' from 'master', open your browser and go to http://node1:2812 then login as admin/monit. From there you should see a screen listing all of the installed sensors and topologies. They will all be off, if I am not mistaken. Click on 'pcap-topology' then 'Start Service'. Then click on 'pcap' and then 'Start Service'. This will fire up the pcap ingestion where you should see packet data landing in HDFS.

Let me know if that works for you. Thanks!

Re: No data in HDFS

Rising Star

I just remembered that the Monit stuff (aka http://node1:2812) is not in master yet. Its is in a PR 103. You can try what I mentioned after that gets merged in.

Prior to that you can SSH to the host and execute the equivalent scripts at /usr/metron/<version>/bin. That would get the job done too.

Don't have an account?
Coming from Hortonworks? Activate your account here