Support Questions

Find answers, ask questions, and share your expertise

Even when i ran balancer, load one data node is 84%. What should be the reason?

Expert Contributor


@Ram D

1. Can you attached screenshot of namenode ui [showing datanode utilization]

2. Can you paste output $hdfs dfsadmin -report

3. top command output from the datanode which has load.

Expert Contributor

Please find the attachments.nnui1.pngtop-and-dfsadmin-report.txt

Super Guru

@Ram D

What is the value of dfs.balance.bandwidthPerSec? What version of Hadoop are you using? There is also protection built in to limit how many blocks can be moved. This is important because you may have other processes running and moving all the data fast will impact those processes. What is the value of dfs.datanode.balance.max.concurrent.moves? A low value will impact how many threads can concurrently move data.

You can increase concurrency but be mindful of the bandwidth you will use and that it is less than your configured dfs.balance.bandwidthPerSec.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.