Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Even when i ran balancer, load one data node is 84%. What should be the reason?

avatar
Expert Contributor
 
1 ACCEPTED SOLUTION

avatar
4 REPLIES 4

avatar
Super Guru
@Ram D

1. Can you attached screenshot of namenode ui [showing datanode utilization]

2. Can you paste output $hdfs dfsadmin -report

3. top command output from the datanode which has load.

avatar
Expert Contributor

Please find the attachments.nnui1.pngtop-and-dfsadmin-report.txt

avatar
Super Guru

@Ram D

What is the value of dfs.balance.bandwidthPerSec? What version of Hadoop are you using? There is also protection built in to limit how many blocks can be moved. This is important because you may have other processes running and moving all the data fast will impact those processes. What is the value of dfs.datanode.balance.max.concurrent.moves? A low value will impact how many threads can concurrently move data.

You can increase concurrency but be mindful of the bandwidth you will use and that it is less than your configured dfs.balance.bandwidthPerSec.

avatar