Support Questions
Find answers, ask questions, and share your expertise

high CPU loads on 3 datanodes


Rebooted my cluster last week and I am seeing unusually high CPU loads on 3 datanodes. Nothing changed, except the reboot. Load average being 120-150 with the others having a load of less than 5. Looking at top, it seems to be yarn java processes that are running. All nodes are running Java 1.7.0

Running HDP 2.5.3 ambari 2.4.2

37 datanodes all configured the same

  • Dell R515 servers
  • 2 8-core AMD CPU's
  • RHEL 6 64-bit
  • 256GB memory
  • same hard drives

The 3 nodes in question only differ by CPU. They are running AMD Opteron 4386 vs AMD Opteron 4284 on all the other datanodes.

I have been running this cluster for over year with no problems. Any ideas as to why this is happening? I have looked at bios setting to see if I had some configured different but they are all the same


Any skew in the data distribution? If yes, may be the HDFS balancer is of some use