Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

high CPU loads on 3 datanodes

Contributor

Rebooted my cluster last week and I am seeing unusually high CPU loads on 3 datanodes. Nothing changed, except the reboot. Load average being 120-150 with the others having a load of less than 5. Looking at top, it seems to be yarn java processes that are running. All nodes are running Java 1.7.0

Running HDP 2.5.3 ambari 2.4.2

37 datanodes all configured the same

  • Dell R515 servers
  • 2 8-core AMD CPU's
  • RHEL 6 64-bit
  • 256GB memory
  • same hard drives

The 3 nodes in question only differ by CPU. They are running AMD Opteron 4386 vs AMD Opteron 4284 on all the other datanodes.

I have been running this cluster for over year with no problems. Any ideas as to why this is happening? I have looked at bios setting to see if I had some configured different but they are all the same

1 REPLY 1

Any skew in the data distribution? If yes, may be the HDFS balancer is of some use

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.