How to connect to remote Hbase using JAVA API

[#1]

Hi,

I am trying to connect to remote Hbase from local system using JAVA API but the same code is working when I run the code from edge node connecting to name node.

Could anyone suggest what changes need to be done on the /etc/hosts file in order to connect to remote Hbase.

Please find the error below:

INFO zookeeper.ZooKeeper: Initiating client connection, connectString=en01.com:2181 sessionTimeout=90000 watcher=hconnection-0xba8d91c0x0, quorum=en01com:2181, baseZNode=/hbase
INFO zookeeper.ClientCnxn: Opening socket connection to server en01com/10.1.20.30:2181. Will not attempt to authenticate using SASL (unknown error)
INFO zookeeper.ClientCnxn: Socket connection established to en01.com/ip:2181, initiating session
INFO zookeeper.ClientCnxn: Session establishment complete on server en01.com/ip:2181, sessionid = 0x65d19392d696c4a, negotiated timeout = 60000

org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions:
Tue Jul 18 10:00:53 CDT 2017, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=68147: row 'tbl_name,,99999999999999' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=dn03.com,60020,1499366811145, seqNum=0

at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:264)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:199)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:56)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
at org.apache.hadoop.hbase.client.ClientSmallReversedScanner.next(ClientSmallReversedScanner.java:145)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1200)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1109)
at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:293)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:131)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:56)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:287)
at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:267)
at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:139)
at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:134)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:823)
at com.seagate.dars.imagedefectanalysis.dao.HBaseDao.saveFiles2FS(HBaseDao.java:177)
at com.seagate.dars.imagedefectanalysis.ImageDefectAnalysis.main(ImageDefectAnalysis.java:40)
Caused by: java.net.SocketTimeoutException: callTimeout=60000, callDuration=68147: row 'tbl_name,,99999999999999' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=dn03.com,60020,1499366811145, seqNum=0
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:159)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:294)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:275)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.UnknownHostException: unknown host: dn03.com
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.<init>(RpcClientImpl.java:296)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.createConnection(RpcClientImpl.java:129)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.getConnection(RpcClientImpl.java:1278)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1152)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:216)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:300)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:31751)
at org.apache.hadoop.hbase.client.ClientSmallScanner$SmallScannerCallable.call(ClientSmallScanner.java:176)
at org.apache.hadoop.hbase.client.ClientSmallScanner$SmallScannerCallable.call(ClientSmallScanner.java:155)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
... 6 more

[#2]

HBase API calls would involve connecting to every HBase service role host on the cluster from the host you are executing on. This requires proper resolution available to discover all RegionServer and Master hostnames.

In your case, your client host is able to resolve the passed ZK hostname of "en01com", but it must also be able to resolve every Master/RS host such as dn03.com.

If you do not rely on a DNS backend to do this for you, your /etc/hosts file must carry every cluster host's entry in the below form:

IP FQDN OptionalShortName

[#3]

All hostnames of the cluster are present in the /etc/hosts/ as below on the "en01".But the dn03 is not getting recognized from the client application.

127.0.0.1 localhost
::1 localhost6

ip.30 en01.nrm.minn.seagate.com en01
ip.31 en02.nrm.minn.seagate.com en02
ip.3 nn01.com nn01
ip.4 nn02.com nn02
ip.5 dn01.com dn01
ip.6 dn02.com dn02
ip.7 dn03.com dn03
ip.8 dn04.com dn04
ip.9 dn05.com dn05
ip.10 dn06.com dn06
ip.11 dn07.com dn07
ip.12 dn08.com dn08
ip.13 dn09.com dn09
ip.14 dn10.com dn10

[#4]

The client application(local system) is connecting to the edge node and /etc/hosts/ on the edge node has all the hostnames as below. Do I need to change anything in this hosts file for my local system to be able to connect to the hbase service on the cluster?

en01 and nn01 has zookeeper , hbase master on nn01 , regionservers are all the datanodes.

Could you please help me in fixing this issue?

127.0.0.1 localhost
::1 localhost6

ip.30 en01.com en01
ip.31 en02.com en02
ip.3 nn01.com nn01
ip.4 nn02.com nn02
ip.5 dn01.com dn01
ip.6 dn02.com dn02
ip.7 dn03.com dn03
ip.8 dn04.com dn04
ip.9 dn05.com dn05
ip.10 dn06.com dn06
ip.11 dn07.com dn07
ip.12 dn08.com dn08
ip.13 dn09.com dn09
ip.14 dn10.com dn10

[#5]

No this didnt solve my issue.