Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Question on HDFS rebalance

avatar
Expert Contributor

HDFS rebalance could be performed from command line or through Ambari UI. Which one do you suggest ? I would like to know the status of the rebalance or logs with error message in case if it fails

1 ACCEPTED SOLUTION

avatar

@Kumar Veerappan

- Via Ambari or Command line. In both ways the same command will be used. The actual command will be : https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#balancer

hdfs balancer -threshold

.

- Ambari will use the following python script and the command: https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/HDFS/2....

def startRebalancingProcess(threshold):
  rebalanceCommand = 'hdfs balancer -threshold %s' % threshold
  return ['cmd', '/C', rebalanceCommand]

- Via command line you will get some additional options to be passed the the "hdfs balancer" command which will allow to get more control on it.

 hdfs balancer
          [-threshold <threshold>]
          [-policy <policy>]
          [-exclude [-f <hosts-file> | <comma-separated list of hosts>]]
          [-include [-f <hosts-file> | <comma-separated list of hosts>]]
          [-idleiterations <idleiterations>]

.

View solution in original post

1 REPLY 1

avatar

@Kumar Veerappan

- Via Ambari or Command line. In both ways the same command will be used. The actual command will be : https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#balancer

hdfs balancer -threshold

.

- Ambari will use the following python script and the command: https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/HDFS/2....

def startRebalancingProcess(threshold):
  rebalanceCommand = 'hdfs balancer -threshold %s' % threshold
  return ['cmd', '/C', rebalanceCommand]

- Via command line you will get some additional options to be passed the the "hdfs balancer" command which will allow to get more control on it.

 hdfs balancer
          [-threshold <threshold>]
          [-policy <policy>]
          [-exclude [-f <hosts-file> | <comma-separated list of hosts>]]
          [-include [-f <hosts-file> | <comma-separated list of hosts>]]
          [-idleiterations <idleiterations>]

.