Metron Connection failed [Errno 111] error on components

[#1]

Dear Metron Community,

I have setup metron full-dev on single node, all components went up normally but when I added some services all components failed and when I checked the alerts it shows Connection failed - Error No 111 on node1

Please help me here I have attached screenshots

[#2 · Accepted solution]

Hi Gaurav,

When you are running all on a single node it is very common that multiple services are fighting for (scarce) available memory. Services like Hbase tend to fail when this happens.

You could try to be selective on what services you run concurrently. For instance if you just want to run Metron services, shut down Ambari Metrics, Oozie, Spark, Zeppelin and maybe Yarn and Hive cause you don't need it. You have to compromise when running single node.

For Metron make sure HDFS, Zookeeper, HBase, Kafka, Storm, ES and Kibana are up and OK.

[#3]

Hey Jasper,

Thank you for your reply, I have ingested logs through NiFi and it is showing in metron management UI but I cant see them on Kibana dashboard how do I configure them through ES, what is the ideal configuration to setup Metron on single node? @Jasper

[#4]

@Gaurav Bapat

On single node just spin up HDFS, Zookeeper, Kafka, Storm, ES and Kibana (and Metron). HBase is only necessary when you do lookups during the enrichment topology. Shut down all other services.

Ambari will be helpful in settings the memory allocations based on what is available to the node. Sometimes you can set them even lower then the recommended settings.

When the VM has less than 8 GB available you will have a real hard time to run Metron at all.

Although it is fairly dated by now, this tutorial would still take you through the required steps to see events pop up in Kibana:

https://cwiki.apache.org/confluence/display/METRON/2016/04/25/Metron+Tutorial+-+Fundamentals+Part+1%...

[#5]

@Jasper

Thank you for your reply, how do I configure Kafka and Zookeeper ??

I have followed the above tutorial and set up KAFKA & ZOOKEEPER to node1:6667 and node1:2181

But I am still not able to get the data to KAFKA and then to KIBANA

Please help & Thanks!!

[#6]

@Gaurav Bapat

Your Nifi setup looks weird. Why do you have 2 PutKafka processors? Look for error messages on the PutKafka processor. Those will tell why the syslog event are not making it into the Kafka topic. (you created a target topic, right?)

I think that at the time of writing that tutorial, the parsing topology was directly writing/indexing into ES. That is not the case anymore in newer versions of Metron. You will have to spin up the indexing topology as well for the same effect, and have indexing source from the Kafka topic that you have as a destination Kafka topic for the parsing topology. You can set that up via Ambari.

[#7]

I have my logs coming into Metron Alerts UI but I cant see them in Kibana, I dont have any tutorial on how do I integrate my logs into the dashboard. Do you have any tutotrial on how do I get them in the dashboard, I cant find any option in ambari and my logs mappings are also empty.

I can see my topic in Storm UI and it is emitting but it has yellow status on port 9300.

Also I dont have my logs as source and I can only see Bro & Snort and not my log as a source.

Thanks for your reply!

[#8]

@Jasper

I am not understanding after getting logs into KAFKA why it is not showing up on KIBANA, I cant find any indexing topology or parsing topology in Ambari, nor do I find any tutorial for the same.

Do you have any solution for this, and then I have to apply Machine Learning on the logs and do analytics

Cant find any way through, need help & Thank You!!

[#9]

@Jasper

Which version of Metron would you recommend me to use so that I can get the logs directly in ES & KIBANA?

[#10]

@Jasper

Which version of Metron would you recommend me to use so that I can get the logs directly in ES & KIBANA?