How do I connect to HBase on Sandbox VM from host computer?

[#1]

I have setup the HDP Sandbox 2.3.2 under VirtualBox running on my Mac. I am able to interact with HBase from inside the VM without any problem. However, when I try to connect via Java from my host, the connection fails. I have forwarded port 16000 and created a "sandbox.hortonworks.com" entry in /etc/hosts (similar to this post). Here is my Java code:

    Configuration conf = HBaseConfiguration.create();
    conf.set("fs.defaultFS", "hdfs://127.0.0.1:8020/");
    conf.set("hbase.zookeeper.property.clientPort", "2181");
    conf.set("hbase.zookeeper.quorum", "sandbox.hortonworks.com");
    conf.set("zookeeper.znode.parent", "/hbase-unsecure");
    HBaseAdmin.checkHBaseAvailable(conf);

Update:

A more helpful exception than the one below:

Master is not running: org.apache.hadoop.hbase.MasterNotRunningException: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.

Not sure why '/hbase' is being used, when I set "zookeeper.znode.parent" to "/hbase-unsecure" explicitly!

Update #2:

Running the code on the Sandbox gives exactly the same error. Any help?

Old: and I get this exception:

org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Call to sandbox.hortonworks.com/127.0.0.1:16000 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Connection to sandbox.hortonworks.com/127.0.0.1:16000 is closing. Call id=0, waitTime=47
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1533)
<snip>

Any suggestions or help? Why would HBase be closing my connection?

[#2 · Accepted solution]

Instead of specifying the config options, you should ALWAYS copy the /etc/hbase/conf folder and add that to your application classpath so that hbase-site.xml gets picked up from the classpath. See https://community.hortonworks.com/articles/4091/hbase-client-application-best-practices.html.

The other thing is that you should open up other ports, 16000, 16030, 16010, 16020, 2181. See http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0-Win/bk_HDP_Install_Win/content/ref-79239257-....

[#3]

@Enis

Thanks for the tip! Worked perfectly.

[#4]

20-05-04 04:25:52 DEBUG ClientCnxn:818 - Reading reply sessionid:0x171dc5f77f00040, packet:: clientPath:null serverPath:null finished:false header:: 23,3 replyHeader:: 23,1099,0 request:: '/hbase-unsecure,F response:: s{116,116,1477381693974,1477381693974,0,24,0,0,0,18,966}
2020-05-04 04:25:52 DEBUG ClientCnxn:717 - Got ping response for sessionid: 0x171dc5f77f00040 after 4ms
2020-05-04 04:25:52 DEBUG ClientCnxn:818 - Reading reply sessionid:0x171dc5f77f00040, packet:: clientPath:null serverPath:null finished:false header:: 24,4 replyHeader:: 24,1099,0 request:: '/hbase-unsecure/master,F response:: #ffffffff000146d61737465723a3136303030ffffffe330ffffff86ffffffafffffffd2352bffffffc850425546a23a1773616e64626f782e686f72746f6e776f726b732e636f6d10ffffff807d18ffffff95ffffffcaffffffbdffffffe3ffffff9d2e10018ffffff8a7d,s{938,938,1588541230259,1588541230259,0,0,0,104106568998453260,71,0,938}
2020-05-04 04:25:52 DEBUG RpcClientImpl:351 - Use SIMPLE authentication for service MasterService, sasl=false
2020-05-04 04:25:52 DEBUG RpcClientImpl:716 - Connecting to sandbox.hortonworks.com/127.0.0.1:16000
2020-05-04 04:25:54 INFO RpcRetryingCaller:132 - Call exception, tries=10, retries=35, started=64734 ms ago, cancelled=false, msg=

[#5]

@Enis Thanks for the input. I am running into the same problem. But, I copied and added the hbase-site.xml to my mac host laptop. Are other file needed as well within (under like: /etc/hbase/conf/core-site.xml)And, now I am running into :

LOG of relevant properties set correctly:

conf.Configuration (Configuration.java:getConfResourceAsInputStream(2355)) - found resource hbase-site.xml at file:/etc/hbase/conf/hbase-site.xml 2016-09-27 12:21:22,651 INFO [main] hbasetest.ConnectHBase (ConnectHBase.java:main(61)) - hbase.zookeeper.quorum:/hbase-unsecure 2016-09-27 12:21:22,651 INFO [main] hbasetest.ConnectHBase (ConnectHBase.java:main(62)) - hbase.zookeeper.quorum:hdfs://sandbox.hortonworks.com:8020 2016-09-27 12:21:22,651 INFO [main] hbasetest.ConnectHBase (ConnectHBase.java:main(63)) - hbase.zookeeper.quorum:2181

Exception:

<code>

where applicable 2016-09-27 12:21:28,045 INFO [main] zookeeper.RecoverableZooKeeper (RecoverableZooKeeper.java:<init>(120)) - Process identifier=hconnection-0x22c86919 connecting to ZooKeeper ensemble=sandbox.hortonworks.com:2181 2016-09-27 12:21:31,461 INFO [main] client.ConnectionManager$HConnectionImplementation (ConnectionManager.java:closeZooKeeperWatcher(1682)) - Closing zookeeper sessionid=0x1569fdbdb97003f Exception in thread "main" org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.net.ConnectException: Connection refused at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1539) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1559) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1710) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.isMasterRunning(ConnectionManager.java:928) at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:2877) at com.plantronics.data.storm.hbasetest.ConnectHBase.main(ConnectHBase.java:68) Caused by: com.google.protobuf.ServiceException: java.net.ConnectException: Connection refused at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:223) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287) at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:53657) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1570) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1508) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1530) ... 5 more Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupConnection(RpcClientImpl.java:410) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:716) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:887) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:856) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1200) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213) ... 10 more Disconnected from the target VM, address: '127.0.0.1:51389', transport: 'socket'

</code>

Wondering if this is port opening issue. Is there a horton help page similar to this for

Seeing a exception for connection opening here:

java.nio.channels.SocketChannel[connection-pending remote=sandbox.hortonworks.com/192.168.56.101:16000]

sandbox.hortonworks.com 16000 (Not able to connect here)

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0-Win/bk_HDP_Install_Win/content/ref-79239257-....

for mac/linux?

[#6]

@Daniel Hendrix

Make sure HBase is running. 16000 port is for ranger KMS so not whats the use of 16000 in this case.

Please follow this article for the java example setup