Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Question on HDFS rebalance

Solved Go to solution

Question on HDFS rebalance

Expert Contributor

HDFS rebalance could be performed from command line or through Ambari UI. Which one do you suggest ? I would like to know the status of the rebalance or logs with error message in case if it fails

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Question on HDFS rebalance

@Kumar Veerappan

- Via Ambari or Command line. In both ways the same command will be used. The actual command will be : https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#balancer

hdfs balancer -threshold

.

- Ambari will use the following python script and the command: https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/HDFS/2....

def startRebalancingProcess(threshold):
  rebalanceCommand = 'hdfs balancer -threshold %s' % threshold
  return ['cmd', '/C', rebalanceCommand]

- Via command line you will get some additional options to be passed the the "hdfs balancer" command which will allow to get more control on it.

 hdfs balancer
          [-threshold <threshold>]
          [-policy <policy>]
          [-exclude [-f <hosts-file> | <comma-separated list of hosts>]]
          [-include [-f <hosts-file> | <comma-separated list of hosts>]]
          [-idleiterations <idleiterations>]

.

1 REPLY 1

Re: Question on HDFS rebalance

@Kumar Veerappan

- Via Ambari or Command line. In both ways the same command will be used. The actual command will be : https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#balancer

hdfs balancer -threshold

.

- Ambari will use the following python script and the command: https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/HDFS/2....

def startRebalancingProcess(threshold):
  rebalanceCommand = 'hdfs balancer -threshold %s' % threshold
  return ['cmd', '/C', rebalanceCommand]

- Via command line you will get some additional options to be passed the the "hdfs balancer" command which will allow to get more control on it.

 hdfs balancer
          [-threshold <threshold>]
          [-policy <policy>]
          [-exclude [-f <hosts-file> | <comma-separated list of hosts>]]
          [-include [-f <hosts-file> | <comma-separated list of hosts>]]
          [-idleiterations <idleiterations>]

.

Don't have an account?
Coming from Hortonworks? Activate your account here