Skip to main content
desind
desind
Gold ⭐️
May 15, 2019
Solved

HDFS diskbalancer

Hello, 

 

Can i run HDFS disk balancer for multiple hosts simultaneously. I tried it and it throws an error which is misleading as already one iteration is in progress.

 

19/05/15 07:19:24 ERROR tools.DiskBalancerCLI: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.diskbalancer.DiskBalancerException): Disk Balancer is not enabled.

 

Thank you,

Des

Harsh J
Harsh JAnswer
Gold ⭐️⭐️
May 16, 2019
The Disk Balancer sub-system is local to each DataNode and can be triggered
on distinct hosts in parallel.

The only time you should receive that exception is if the targeted DN's
hdfs-site.xml does not carry the property that enables disk balancer, or
when the DataNode is mid-shutdown/restart.

How have you configured disk balancer for your cluster? Did you follow the
configuration approach presented at
https://blog.cloudera.com/blog/2016/10/how-to-use-the-new-hdfs-intra-datanode-disk-balancer-in-apache-hadoop/?
What is your CDH and CM version?
View in thread

1 reply

Harsh J
Gold ⭐️⭐️
Harsh JAnswer
Gold ⭐️⭐️
May 16, 2019
The Disk Balancer sub-system is local to each DataNode and can be triggered
on distinct hosts in parallel.

The only time you should receive that exception is if the targeted DN's
hdfs-site.xml does not carry the property that enables disk balancer, or
when the DataNode is mid-shutdown/restart.

How have you configured disk balancer for your cluster? Did you follow the
configuration approach presented at
https://blog.cloudera.com/blog/2016/10/how-to-use-the-new-hdfs-intra-datanode-disk-balancer-in-apache-hadoop/?
What is your CDH and CM version?
Translate
    Terms & ConditionsAccessibility statement