Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unable to use hbase shell from HA server

Unable to use hbase shell from HA server

New Contributor

Hi,

 

I have enabled HA in a server and from then i was unable to access hbase shell from the server which is HA server.

 

HA server has hbase backup master role

and its log has below lines repeatedly

7:41:15.819 AM INFO org.apache.zookeeper.ClientCnxn
Opening socket connection to server lonplhadooappu12.uk.db.com/172.17.239.95:2181. Will not attempt to authenticate using SASL (unknown error)
7:41:15.819 AM INFO org.apache.zookeeper.ClientCnxn
Socket connection established to lonplhadooappu12.uk.db.com/172.17.239.95:2181, initiating session
7:41:15.820 AM INFO org.apache.zookeeper.ClientCnxn
Unable to read additional data from server sessionid 0x14be7c8d4f29dca, likely server has closed socket, closing socket connection and attempting reconnect
7:41:16.989 AM INFO org.apache.zookeeper.ClientCnxn
Opening socket connection to server lonplhadooappu14.uk.db.com/172.17.239.97:2181. Will not attempt to authenticate using SASL (unknown error)
7:41:16.989 AM INFO org.apache.zookeeper.ClientCnxn
Socket connection established to lonplhadooappu14.uk.db.com/172.17.239.97:2181, initiating session

 

and also below warning sometimes:

 

Session 0x14be7c8d4f29dca for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)

 

 

And zookeeper log holds below warning at the same time:

 

caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x34cea290bf92364, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
at java.lang.Thread.run(Thread.java:745)

 

I checked apache community and according to them this issue should be fixed with zookeeper 3.4.5.

i am using jdk8  and CDH 5.3 which has version 3.4.5 of zookeeper.

3 REPLIES 3

Re: Unable to use hbase shell from HA server

Contributor

Hi,

 

The ZooKeeper error you see can be ignored. Basically, it's loged because a connection has been made to HBase and has been closed before HBase closed it. Like when you exit the shell.

 

Are you able to connect to HBase shell from any of the othe nodes? Like from a RegionServer? From the other master? Etc.

 

Have you re-deployed the clients configs?

 

Thanks,

 

JMS

Re: Unable to use hbase shell from HA server

New Contributor

Hi,

 

Thanks for the reply, now i am able to use hbase shell after changing roles on the server, i moved hbase master(backup) to other node and moved thrift server to Standby namenode.

 

it is working well and didnt face issue later. but still didnt understand why the role didnt work :(

 

earlier i was able to access hbase shell from all the servers except the standyby namenode. 

 

Thanks 

Shashank

Highlighted

Re: Unable to use hbase shell from HA server

Contributor

Hi Shashank,

 

To access the shell, you need to make sure you have at least one HBase role deployed on the server you are trying to use to access the shell.

You said it was failing from the Standby namenode. Was there an HBase gateway or master deployed on this serer? If not, that might explain.

 

If you add a new role, don't forget to deploy the client configuration.

 

JM