Support Questions

Find answers, ask questions, and share your expertise
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

All node managers down

Expert Contributor

My resource managers are active and so is Job history server. All my worker nodes had been exiting randomly for some time but used to restart automatically. today, all my node managers are down. what could be the reason? My worker nodes are typical with Hdfs And yarn on them. hdfs is running fine. what does it indicate when all node managers are down? There was no unusual load on servers. also, if i restart them, it still goes down. please suggest what could cause this?


New Contributor

Check network connection between node managers and cloudera manager, this could a network issue. try to do a 100 MB file transfer between trouble hosts and healthy hosts, compare time between them.

if file transfer between nodes (with node manager down) is taking longer than expected, you have to contact your network team to check network switch connecting those nodes.

Expert Contributor

But in that case, HDFS would be down as well, No? HDFS is installed on same servers as Node managers are and HDFS is working fine without any warnings or errors

Expert Contributor

Also, Node manager continue to exit even if it is on same node as CM

New Contributor

i faced similar issue with 'impalad'. where there was issue with network switch issue.

i suggest its worth trying.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.