question HDFS diskbalancer in Support Questions

HDFS diskbalancer

desind — Wed, 15 May 2019 11:25:15 GMT

Hello,

Can i run HDFS disk balancer for multiple hosts simultaneously. I tried it and it throws an error which is misleading as already one iteration is in progress.

19/05/15 07:19:24 ERROR tools.DiskBalancerCLI: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.diskbalancer.DiskBalancerException): Disk Balancer is not enabled.

Thank you,

Des

Re: HDFS diskbalancer

Harsh J — Thu, 16 May 2019 01:52:15 GMT

The Disk Balancer sub-system is local to each DataNode and can be triggered
on distinct hosts in parallel.

The only time you should receive that exception is if the targeted DN's
hdfs-site.xml does not carry the property that enables disk balancer, or
when the DataNode is mid-shutdown/restart.

How have you configured disk balancer for your cluster? Did you follow the
configuration approach presented at
https://blog.cloudera.com/blog/2016/10/how-to-use-the-new-hdfs-intra-datanode-disk-balancer-in-apache-hadoop/?
What is your CDH and CM version?