Question on HDFS rebalance

kums — Wed, 07 Dec 2016 00:12:43 GMT

HDFS rebalance could be performed from command line or through Ambari UI. Which one do you suggest ? I would like to know the status of the rebalance or logs with error message in case if it fails

Re: Question on HDFS rebalance

Former Member — Wed, 07 Dec 2016 00:21:17 GMT

@Kumar Veerappan

- Via Ambari or Command line. In both ways the same command will be used. The actual command will be : https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#balancer

hdfs balancer -threshold

- Ambari will use the following python script and the command: https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py#L315-L317

def startRebalancingProcess(threshold):
  rebalanceCommand = 'hdfs balancer -threshold %s' % threshold
  return ['cmd', '/C', rebalanceCommand]

- Via command line you will get some additional options to be passed the the "hdfs balancer" command which will allow to get more control on it.

 hdfs balancer
          [-threshold <threshold>]
          [-policy <policy>]
          [-exclude [-f <hosts-file> | <comma-separated list of hosts>]]
          [-include [-f <hosts-file> | <comma-separated list of hosts>]]
          [-idleiterations <idleiterations>]

question Question on HDFS rebalance in Archives of Support Questions (Read Only)

Question on HDFS rebalance

Re: Question on HDFS rebalance