Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

hadoop balancer is running slow

hadoop balancer is running slow

New Contributor

Hi,

My cluster has machines has 10GB/sec bandwidth speed.

I am trying to run balancer. The data balancing is so slow. It has balanced only 5TB in 10 hours. Could you please suggest the good recommendations for the balancer asap so that I can schedule for this weekend. Currently I am interested in the below properties.

1. Bandwidth for the balancer

2. dfs.datanode.balance.bandwidthPerSec

3. dfs.balancer.max-size-to-move

4. dfs.datanode.balance.max.concurrent.moves

5. dfs.balancer.moverThreads

Please let us know how to set these properties to get high speed. I also see sometimes socket timeout exception in the balancer logs

My current socket timeout exception is 1min.. Please suggest the recommendations asap.

Don't have an account?
Coming from Hortonworks? Activate your account here