Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark to HBase Connectivity Issue

Highlighted

Spark to HBase Connectivity Issue

Contributor

Hi Team,

We are using HDP 2.6.5 and we are trying to test Spark and HBase integration but we are facing issues while testing. We are following the link https://github.com/hortonworks-spark/shc.

Everytime we are facing connection refused from the Zookeeper.

Please help on this issue as we are stuck. We need to build DF over HBase tables. Note that the HDP cluster is Kerberized.

Below is the screen dump:

18/09/05 08:47:38 WARN ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125)
18/09/05 08:47:38 WARN ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125)
18/09/05 08:47:38 WARN RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
18/09/05 08:47:39 WARN ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
1 REPLY 1

Re: Spark to HBase Connectivity Issue

Possibly transient ZooKeeper, quorum=localhost:2181

Is ZooKeeper actually running on localhost? If you are not explicitly providing the ZK quorum in your code, it sounds like you would need to make sure the correct hbase-site.xml configuration file on the classpath for SHC.

Don't have an account?
Coming from Hortonworks? Activate your account here