HDFS Balancer slow in CDH5.3.2

[#1]

Hi All,

We recently added 11 new datanodes to CDH5.3.2 cluster which has 20 datanodes.

I see HDFS Balancer slowly moving blocks say avg 4000 blocks peer 10 mins. I' updated Balancer Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml

with 

<property>
<name>dfs.balance.bandwidthPerSec</name>
<value>3048576000</value>
</property>

still i see alomst same speed, i checked network graphs also

Is there any other setting required?

Thanks,

Venu

[#2 · Accepted solution]

I opned case with cloudera and found solution

Add following settings in Balancer Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml

<property>
<name>dfs.datanode.balance.max.concurrent.moves</name>
<value>300</value>
</property>

Thanks,

Venu

[#3]

I'm pretty sure that this configuration should go not to Balancer "Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml", but in all Datanodes "Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml" and DNs should be restarted after configuration changes.

[#4]

It should go to both of those valves with the same value currently, until HDFS-10289 gets done in future.