Support Questions

Find answers, ask questions, and share your expertise

Hdfs Balancer exits without balancing with error


Hello All,

HDP 2.6.3 Datanodes 3

I'm trying to balance my datanodes by running the below balancer command via CLI, I found that my balancer did not point to any of my Namenodes and also I found the below error.

devcluster is my Cluster name , masternode1dev is my Standby NN.

hdfs balancer -threshold 10

INFO balancer.Balancer: namenodes = [hdfs://devcluster]

INFO retry.RetryInvocationHandler: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): Operation category READ is not supported in state standby

at org.apache.hadoop.hdfs.server.namenode.ha.StandbyState.checkOperation(

at org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.checkOperation(

at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkOperation(

at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getBlocks(

at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlocks(

at org.apache.hadoop.hdfs.protocolPB.NamenodeProtocolServerSideTranslatorPB.getBlocks(

at org.apache.hadoop.hdfs.protocol.proto.NamenodeProtocolProtos$NamenodeProtocolService$2.callBlockingMethod(

at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$

at org.apache.hadoop.ipc.RPC$

at org.apache.hadoop.ipc.Server$Handler$

at org.apache.hadoop.ipc.Server$Handler$

at Method)



at org.apache.hadoop.ipc.Server$

, while invoking NamenodeProtocolTranslatorPB.getBlocks over masternode1dev/xxx.xx.xx.xx:8020. Trying to failover immediately.


@Joshva Peter

Above message "Operation category READ is not supported in state standby" is when the Balancer connects to Standby Namenode and the connection would later failover to other Active Namenode.

Since, there is no error reported. Capture the debug logs as below to analyse the issue.

export HADOOP_BALANCER_OPTS="-Droot.logger=DEBUG,console"

Run the balancer as:

hdfs balancer 'options' 2>&1 | tee /tmp/balancer.log



Output of hdfs balancer 'options'2>&1| tee /tmp/balancer.log

[hdfs@masternode2dev ~]$ cat /tmp/balancer.log Usage: hdfs balancer [-policy <policy>] the balancing policy: datanode or blockpool [-threshold <threshold>] Percentage of disk capacity [-exclude [-f <hosts-file> | <comma-separated list of hosts>]] Excludes the specified datanodes. [-include [-f <hosts-file> | <comma-separated list of hosts>]] Includes only the specified datanodes. [-source [-f <hosts-file> | <comma-separated list of hosts>]] Pick only the specified datanodes as source nodes. [-idleiterations <idleiterations>] Number of consecutive idle iterations (-1 for Infinite) before exit. [-runDuringUpgrade] Whether to run the balancer during an ongoing HDFS upgrade.This is usually not desired since it will not affect used space on over-utilized machines.

Jan 21, 2019 9:29:33 PM Balancing took 316.0 milliseconds

19/01/21 21:29:33 ERROR balancer.Balancer: Exiting balancer due an exception java.lang.IllegalArgumentException: args = [options]

at org.apache.hadoop.hdfs.server.balancer.Balancer$Cli.parse(

at org.apache.hadoop.hdfs.server.balancer.Balancer$


at org.apache.hadoop.hdfs.server.balancer.Balancer.main(