Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

high CPU loads on 3 datanodes

Highlighted

high CPU loads on 3 datanodes

Contributor

Rebooted my cluster last week and I am seeing unusually high CPU loads on 3 datanodes. Nothing changed, except the reboot. Load average being 120-150 with the others having a load of less than 5. Looking at top, it seems to be yarn java processes that are running. All nodes are running Java 1.7.0

Running HDP 2.5.3 ambari 2.4.2

37 datanodes all configured the same

  • Dell R515 servers
  • 2 8-core AMD CPU's
  • RHEL 6 64-bit
  • 256GB memory
  • same hard drives

The 3 nodes in question only differ by CPU. They are running AMD Opteron 4386 vs AMD Opteron 4284 on all the other datanodes.

I have been running this cluster for over year with no problems. Any ideas as to why this is happening? I have looked at bios setting to see if I had some configured different but they are all the same

1 REPLY 1

Re: high CPU loads on 3 datanodes

Any skew in the data distribution? If yes, may be the HDFS balancer is of some use

Don't have an account?
Coming from Hortonworks? Activate your account here