Support Questions
Find answers, ask questions, and share your expertise

Question on HDFS rebalance

Solved Go to solution
Highlighted

Question on HDFS rebalance

Expert Contributor

HDFS rebalance could be performed from command line or through Ambari UI. Which one do you suggest ? I would like to know the status of the rebalance or logs with error message in case if it fails

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Question on HDFS rebalance

@Kumar Veerappan

- Via Ambari or Command line. In both ways the same command will be used. The actual command will be : https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#balancer

hdfs balancer -threshold

.

- Ambari will use the following python script and the command: https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/HDFS/2....

def startRebalancingProcess(threshold):
  rebalanceCommand = 'hdfs balancer -threshold %s' % threshold
  return ['cmd', '/C', rebalanceCommand]

- Via command line you will get some additional options to be passed the the "hdfs balancer" command which will allow to get more control on it.

 hdfs balancer
          [-threshold <threshold>]
          [-policy <policy>]
          [-exclude [-f <hosts-file> | <comma-separated list of hosts>]]
          [-include [-f <hosts-file> | <comma-separated list of hosts>]]
          [-idleiterations <idleiterations>]

.

View solution in original post

1 REPLY 1
Highlighted

Re: Question on HDFS rebalance

@Kumar Veerappan

- Via Ambari or Command line. In both ways the same command will be used. The actual command will be : https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#balancer

hdfs balancer -threshold

.

- Ambari will use the following python script and the command: https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/HDFS/2....

def startRebalancingProcess(threshold):
  rebalanceCommand = 'hdfs balancer -threshold %s' % threshold
  return ['cmd', '/C', rebalanceCommand]

- Via command line you will get some additional options to be passed the the "hdfs balancer" command which will allow to get more control on it.

 hdfs balancer
          [-threshold <threshold>]
          [-policy <policy>]
          [-exclude [-f <hosts-file> | <comma-separated list of hosts>]]
          [-include [-f <hosts-file> | <comma-separated list of hosts>]]
          [-idleiterations <idleiterations>]

.

View solution in original post

Don't have an account?