Support Questions

Find answers, ask questions, and share your expertise

ambari hdfs rebalance failed (RpcNoSuchMethodException)

avatar
Contributor
Hi, All

I met a problem when rebalancing hdfs as follows: 
ambari : 2.1.2.1

Is there anyone have an idea about this? 

and where is the script of 'start-balancer.sh' ? 

.....
16/05/23 11:01:21 INFO balancer.Balancer: Need to move 14.26 TB to make the cluster balanced.
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RpcNoSuchMethodException): Unknown method isUpgradeFinalized called on org.apache.hadoop.hdfs.server.protocol.NamenodeProtocol protocol.
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:575)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
.  Exiting ...
May 23, 2016 11:01:21 AM          0                  0 B                -1 B               -1 B
May 23, 2016 11:01:21 AM Balancing took 1.618 seconds

Thanks in advance 
Park
1 ACCEPTED SOLUTION

avatar

@Young-Geun Park

Looks like you are hitting a bug.

https://issues.apache.org/jira/browse/AMBARI-13373

WORKAROUND:

Use the configs.sh utility on the Ambari Server to delete the extra property.

Important: Only perform this procedure if your cluster is running NameNode HA and you are experiencing the HDFS balancer issue.

/var/lib/ambari-server/resources/scripts/configs.sh -u <admin.user> -p <admin.password> delete

<ambari.server> <cluster.name> hdfs-site “dfs.namenode.rpc-address”

Where

admin.user and

admin.password are credentials for an Ambari Administrator,

ambari.server is the Ambari Server host and

cluster.name is the name of your cluster.

View solution in original post

3 REPLIES 3

avatar

@Young-Geun Park

Looks like you are hitting a bug.

https://issues.apache.org/jira/browse/AMBARI-13373

WORKAROUND:

Use the configs.sh utility on the Ambari Server to delete the extra property.

Important: Only perform this procedure if your cluster is running NameNode HA and you are experiencing the HDFS balancer issue.

/var/lib/ambari-server/resources/scripts/configs.sh -u <admin.user> -p <admin.password> delete

<ambari.server> <cluster.name> hdfs-site “dfs.namenode.rpc-address”

Where

admin.user and

admin.password are credentials for an Ambari Administrator,

ambari.server is the Ambari Server host and

cluster.name is the name of your cluster.

avatar
Contributor

thanks for your reply in advance.

https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.0.0/bk_Ambari_Users_Guide/content/_modify_hdfs_...

Is there any impact on hadoop operations after "config.sh delete ... hdfs-site “dfs.namenode.rpc-address" command runs?

avatar
@Young-Geun Park

No, we need to restart couple of services I guess.