Spark to HBase Connectivity Issue

[#1]

Hi Team,

We are using HDP 2.6.5 and we are trying to test Spark and HBase integration but we are facing issues while testing. We are following the link https://github.com/hortonworks-spark/shc.

Everytime we are facing connection refused from the Zookeeper.

Please help on this issue as we are stuck. We need to build DF over HBase tables. Note that the HDP cluster is Kerberized.

Below is the screen dump:

18/09/05 08:47:38 WARN ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
  at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
  at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
  at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125)
18/09/05 08:47:38 WARN ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
  at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
  at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
  at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125)
18/09/05 08:47:38 WARN RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
18/09/05 08:47:39 WARN ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused

[#2]

Possibly transient ZooKeeper, quorum=localhost:2181

Is ZooKeeper actually running on localhost? If you are not explicitly providing the ZK quorum in your code, it sounds like you would need to make sure the correct hbase-site.xml configuration file on the classpath for SHC.