Reply
New Contributor
Posts: 3
Registered: ‎04-28-2015

Unable to use hbase shell from HA server

Hi,

 

I have enabled HA in a server and from then i was unable to access hbase shell from the server which is HA server.

 

HA server has hbase backup master role

and its log has below lines repeatedly

7:41:15.819 AM INFO org.apache.zookeeper.ClientCnxn
Opening socket connection to server lonplhadooappu12.uk.db.com/172.17.239.95:2181. Will not attempt to authenticate using SASL (unknown error)
7:41:15.819 AM INFO org.apache.zookeeper.ClientCnxn
Socket connection established to lonplhadooappu12.uk.db.com/172.17.239.95:2181, initiating session
7:41:15.820 AM INFO org.apache.zookeeper.ClientCnxn
Unable to read additional data from server sessionid 0x14be7c8d4f29dca, likely server has closed socket, closing socket connection and attempting reconnect
7:41:16.989 AM INFO org.apache.zookeeper.ClientCnxn
Opening socket connection to server lonplhadooappu14.uk.db.com/172.17.239.97:2181. Will not attempt to authenticate using SASL (unknown error)
7:41:16.989 AM INFO org.apache.zookeeper.ClientCnxn
Socket connection established to lonplhadooappu14.uk.db.com/172.17.239.97:2181, initiating session

 

and also below warning sometimes:

 

Session 0x14be7c8d4f29dca for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)

 

 

And zookeeper log holds below warning at the same time:

 

caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x34cea290bf92364, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
at java.lang.Thread.run(Thread.java:745)

 

I checked apache community and according to them this issue should be fixed with zookeeper 3.4.5.

i am using jdk8  and CDH 5.3 which has version 3.4.5 of zookeeper.

Cloudera Employee
Posts: 35
Registered: ‎07-08-2013

Re: Unable to use hbase shell from HA server

Hi,

 

The ZooKeeper error you see can be ignored. Basically, it's loged because a connection has been made to HBase and has been closed before HBase closed it. Like when you exit the shell.

 

Are you able to connect to HBase shell from any of the othe nodes? Like from a RegionServer? From the other master? Etc.

 

Have you re-deployed the clients configs?

 

Thanks,

 

JMS

New Contributor
Posts: 3
Registered: ‎04-28-2015

Re: Unable to use hbase shell from HA server

Hi,

 

Thanks for the reply, now i am able to use hbase shell after changing roles on the server, i moved hbase master(backup) to other node and moved thrift server to Standby namenode.

 

it is working well and didnt face issue later. but still didnt understand why the role didnt work :(

 

earlier i was able to access hbase shell from all the servers except the standyby namenode. 

 

Thanks 

Shashank

Highlighted
Cloudera Employee
Posts: 35
Registered: ‎07-08-2013

Re: Unable to use hbase shell from HA server

Hi Shashank,

 

To access the shell, you need to make sure you have at least one HBase role deployed on the server you are trying to use to access the shell.

You said it was failing from the Standby namenode. Was there an HBase gateway or master deployed on this serer? If not, that might explain.

 

If you add a new role, don't forget to deploy the client configuration.

 

JM

Announcements