Hi, I am trying to connect to Hbase table using JAVA application that is running on windows machine.I could establish connection to the table using zookeeper thats running on the edge node and get the region server IP address from the .meta table. Then the client application is trying to connect directly to the region server (data node) from windows machine and throwing the below exceptions. Due to edge node we cannot connect from windows to data node directly and also edge node and other nodes (master and data nodes) are in different network , how can this issue be resolved? 17/08/04 09:18:34 ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries. at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:356) at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:371) at org.apache.hadoop.util.Shell.<clinit>(Shell.java:364) at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:80) at org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1437) at org.apache.hadoop.hbase.HBaseConfiguration.checkDefaultsVersion(HBaseConfiguration.java:67) at org.apache.hadoop.hbase.HBaseConfiguration.addHbaseResources(HBaseConfiguration.java:81) at org.apache.hadoop.hbase.HBaseConfiguration.create(HBaseConfiguration.java:96) at com.seagate.dars.imagedefectanalysis.dao.HBaseDao.saveFiles2FS(HBaseDao.java:138) at com.seagate.dars.imagedefectanalysis.ImageDefectAnalysis.main(ImageDefectAnalysis.java:40) 17/08/04 09:18:34 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 17/08/04 09:18:34 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0xba8d91c connecting to ZooKeeper ensemble=en01.com:2181 17/08/04 09:18:34 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 17/08/04 09:18:34 INFO zookeeper.ZooKeeper: Client environment:host.name=NRM-UabcL003.ad.seagate.com 17/08/04 09:18:34 INFO zookeeper.ZooKeeper: Client environment:java.version=1.8.0_112 17/08/04 09:18:34 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 17/08/04 09:18:34 INFO zookeeper.ZooKeeper: Client environment:java.home=C:\Users\abc\AppData\Local\MyEclipse 2017 CI\binary\com.sun.java.jdk8.win32.x86_64_1.8.0.v112\jre 17/08/04 09:18:34 INFO zookeeper.ZooKeeper: Client environment:java.class.path=C:\Users\abc\renuka\Workspaces\MyEclipse 2017 CI\ImageDefectAnalysis_original\target\classes;C:\Users\abc\.m2\repository\org\apache\hbase\hbase-client\1.0.0\hbase-client-1.0.0.jar;C:\Users\abc\.m2\repository\org\apache\hbase\hbase-annotations\1.0.0\hbase-annotations-1.0.0.jar;C:\Users\abc\AppData\Local\MyEclipse 2017 CI\binary\com.sun.java.jdk8.win32.x86_64_1.8.0.v112\lib\tools.jar;C:\Users\abc\.m2\repository\org\apache\hbase\hbase-common\1.0.0\hbase-common-1.0.0.jar;C:\Users\abc\.m2\repository\org\apache\hbase\hbase-protocol\1.0.0\hbase-protocol-1.0.0.jar;C:\Users\abc\.m2\repository\commons-codec\commons-codec\1.9\commons-codec-1.9.jar;C:\Users\abc\.m2\repository\commons-io\commons-io\2.4\commons-io-2.4.jar;C:\Users\abc\.m2\repository\commons-lang\commons-lang\2.6\commons-lang-2.6.jar;C:\Users\abc\.m2\repository\commons-logging\commons-logging\1.2\commons-logging-1.2.jar;C:\Users\abc\.m2\repository\com\google\guava\guava\12.0.1\guava-12.0.1.jar;C:\Users\abc\.m2\repository\com\google\protobuf\protobuf-java\2.5.0\protobuf-java-2.5.0.jar;C:\Users\abc\.m2\repository\io\netty\netty-all\4.0.23.Final\netty-all-4.0.23.Final.jar;C:\Users\abc\.m2\repository\org\apache\zookeeper\zookeeper\3.4.6\zookeeper-3.4.6.jar;C:\Users\abc\.m2\repository\org\apache\htrace\htrace-core\3.1.0-incubating\htrace-core-3.1.0-incubating.jar;C:\Users\abc\.m2\repository\org\codehaus\jackson\jackson-mapper-asl\1.8.8\jackson-mapper-asl-1.8.8.jar;C:\Users\abc\.m2\repository\org\jruby\jcodings\jcodings\1.0.8\jcodings-1.0.8.jar;C:\Users\abc\.m2\repository\org\jruby\joni\joni\2.1.2\joni-2.1.2.jar;C:\Users\abc\.m2\repository\org\apache\hadoop\hadoop-auth\2.5.1\hadoop-auth-2.5.1.jar;C:\Users\abc\.m2\repository\org\apache\httpcomponents\httpclient\4.2.5\httpclient-4.2.5.jar;C:\Users\abc\.m2\repository\org\apache\directory\server\apacheds-kerberos-codec\2.0.0-M15\apacheds-kerberos-codec-2.0.0-M15.jar;C:\Users\abc\.m2\repository\org\apache\directory\server\apacheds-i18n\2.0.0-M15\apacheds-i18n-2.0.0-M15.jar;C:\Users\abc\.m2\repository\org\apache\directory\api\api-asn1-api\1.0.0-M20\api-asn1-api-1.0.0-M20.jar;C:\Users\abc\.m2\repository\org\apache\directory\api\api-util\1.0.0-M20\api-util-1.0.0-M20.jar;C:\Users\abc\.m2\repository\org\apache\hadoop\hadoop-mapreduce-client-core\2.5.1\hadoop-mapreduce-client-core-2.5.1.jar;C:\Users\abc\.m2\repository\org\apache\hadoop\hadoop-yarn-common\2.5.1\hadoop-yarn-common-2.5.1.jar;C:\Users\abc\.m2\repository\org\apache\hadoop\hadoop-yarn-api\2.5.1\hadoop-yarn-api-2.5.1.jar;C:\Users\abc\.m2\repository\javax\xml\bind\jaxb-api\2.2.2\jaxb-api-2.2.2.jar;C:\Users\abc\.m2\repository\javax\xml\stream\stax-api\1.0-2\stax-api-1.0-2.jar;C:\Users\abc\.m2\repository\io\netty\netty\3.6.2.Final\netty-3.6.2.Final.jar;C:\Users\abc\.m2\repository\com\github\stephenc\findbugs\findbugs-annotations\1.3.9-1\findbugs-annotations-1.3.9-1.jar;C:\Users\abc\.m2\repository\junit\junit\4.11\junit-4.11.jar;C:\Users\abc\.m2\repository\org\hamcrest\hamcrest-core\1.3\hamcrest-core-1.3.jar;C:\Users\abc\.m2\repository\org\apache\hadoop\hadoop-hdfs\2.7.1\hadoop-hdfs-2.7.1.jar;C:\Users\abc\.m2\repository\org\mortbay\jetty\jetty\6.1.26\jetty-6.1.26.jar;C:\Users\abc\.m2\repository\org\mortbay\jetty\jetty-util\6.1.26\jetty-util-6.1.26.jar;C:\Users\abc\.m2\repository\com\sun\jersey\jersey-core\1.9\jersey-core-1.9.jar;C:\Users\abc\.m2\repository\com\sun\jersey\jersey-server\1.9\jersey-server-1.9.jar;C:\Users\abc\.m2\repository\asm\asm\3.1\asm-3.1.jar;C:\Users\abc\.m2\repository\commons-cli\commons-cli\1.2\commons-cli-1.2.jar;C:\Users\abc\.m2\repository\commons-daemon\commons-daemon\1.0.13\commons-daemon-1.0.13.jar;C:\Users\abc\.m2\repository\log4j\log4j\1.2.17\log4j-1.2.17.jar;C:\Users\abc\.m2\repository\javax\servlet\servlet-api\2.5\servlet-api-2.5.jar;C:\Users\abc\.m2\repository\org\codehaus\jackson\jackson-core-asl\1.9.13\jackson-core-asl-1.9.13.jar;C:\Users\abc\.m2\repository\xmlenc\xmlenc\0.52\xmlenc-0.52.jar;C:\Users\abc\.m2\repository\xerces\xercesImpl\2.9.1\xercesImpl-2.9.1.jar;C:\Users\abc\.m2\repository\xml-apis\xml-apis\1.3.04\xml-apis-1.3.04.jar;C:\Users\abc\.m2\repository\org\fusesource\leveldbjni\leveldbjni-all\1.8\leveldbjni-all-1.8.jar;C:\Users\abc\.m2\repository\org\apache\hadoop\hadoop-common\2.7.1\hadoop-common-2.7.1.jar;C:\Users\abc\.m2\repository\org\apache\hadoop\hadoop-annotations\2.7.1\hadoop-annotations-2.7.1.jar;C:\Users\abc\.m2\repository\org\apache\commons\commons-math3\3.1.1\commons-math3-3.1.1.jar;C:\Users\abc\.m2\repository\commons-httpclient\commons-httpclient\3.1\commons-httpclient-3.1.jar;C:\Users\abc\.m2\repository\commons-net\commons-net\3.1\commons-net-3.1.jar;C:\Users\abc\.m2\repository\commons-collections\commons-collections\3.2.1\commons-collections-3.2.1.jar;C:\Users\abc\.m2\repository\javax\servlet\jsp\jsp-api\2.1\jsp-api-2.1.jar;C:\Users\abc\.m2\repository\com\sun\jersey\jersey-json\1.9\jersey-json-1.9.jar;C:\Users\abc\.m2\repository\org\codehaus\jettison\jettison\1.1\jettison-1.1.jar;C:\Users\abc\.m2\repository\com\sun\xml\bind\jaxb-impl\2.2.3-1\jaxb-impl-2.2.3-1.jar;C:\Users\abc\.m2\repository\org\codehaus\jackson\jackson-jaxrs\1.8.3\jackson-jaxrs-1.8.3.jar;C:\Users\abc\.m2\repository\org\codehaus\jackson\jackson-xc\1.8.3\jackson-xc-1.8.3.jar;C:\Users\abc\.m2\repository\net\java\dev\jets3t\jets3t\0.9.0\jets3t-0.9.0.jar;C:\Users\abc\.m2\repository\org\apache\httpcomponents\httpcore\4.1.2\httpcore-4.1.2.jar;C:\Users\abc\.m2\repository\com\jamesmurty\utils\java-xmlbuilder\0.4\java-xmlbuilder-0.4.jar;C:\Users\abc\.m2\repository\commons-configuration\commons-configuration\1.6\commons-configuration-1.6.jar;C:\Users\abc\.m2\repository\commons-digester\commons-digester\1.8\commons-digester-1.8.jar;C:\Users\abc\.m2\repository\commons-beanutils\commons-beanutils\1.7.0\commons-beanutils-1.7.0.jar;C:\Users\abc\.m2\repository\commons-beanutils\commons-beanutils-core\1.8.0\commons-beanutils-core-1.8.0.jar;C:\Users\abc\.m2\repository\org\slf4j\slf4j-api\1.7.10\slf4j-api-1.7.10.jar;C:\Users\abc\.m2\repository\org\slf4j\slf4j-log4j12\1.7.10\slf4j-log4j12-1.7.10.jar;C:\Users\abc\.m2\repository\org\apache\avro\avro\1.7.4\avro-1.7.4.jar;C:\Users\abc\.m2\repository\com\thoughtworks\paranamer\paranamer\2.3\paranamer-2.3.jar;C:\Users\abc\.m2\repository\org\xerial\snappy\snappy-java\1.0.4.1\snappy-java-1.0.4.1.jar;C:\Users\abc\.m2\repository\com\google\code\gson\gson\2.2.4\gson-2.2.4.jar;C:\Users\abc\.m2\repository\com\jcraft\jsch\0.1.42\jsch-0.1.42.jar;C:\Users\abc\.m2\repository\org\apache\curator\curator-client\2.7.1\curator-client-2.7.1.jar;C:\Users\abc\.m2\repository\org\apache\curator\curator-recipes\2.7.1\curator-recipes-2.7.1.jar;C:\Users\abc\.m2\repository\org\apache\curator\curator-framework\2.7.1\curator-framework-2.7.1.jar;C:\Users\abc\.m2\repository\com\google\code\findbugs\jsr305\3.0.0\jsr305-3.0.0.jar;C:\Users\abc\.m2\repository\org\apache\commons\commons-compress\1.4.1\commons-compress-1.4.1.jar;C:\Users\abc\.m2\repository\org\tukaani\xz\1.0\xz-1.0.jar;C:\Users\abc\.m2\repository\org\apache\hadoop\hadoop-core\1.2.1\hadoop-core-1.2.1.jar;C:\Users\abc\.m2\repository\org\apache\commons\commons-math\2.1\commons-math-2.1.jar;C:\Users\abc\.m2\repository\tomcat\jasper-runtime\5.5.12\jasper-runtime-5.5.12.jar;C:\Users\abc\.m2\repository\tomcat\jasper-compiler\5.5.12\jasper-compiler-5.5.12.jar;C:\Users\abc\.m2\repository\org\mortbay\jetty\jsp-api-2.1\6.1.14\jsp-api-2.1-6.1.14.jar;C:\Users\abc\.m2\repository\org\mortbay\jetty\servlet-api-2.5\6.1.14\servlet-api-2.5-6.1.14.jar;C:\Users\abc\.m2\repository\org\mortbay\jetty\jsp-2.1\6.1.14\jsp-2.1-6.1.14.jar;C:\Users\abc\.m2\repository\ant\ant\1.6.5\ant-1.6.5.jar;C:\Users\abc\.m2\repository\commons-el\commons-el\1.0\commons-el-1.0.jar;C:\Users\abc\.m2\repository\hsqldb\hsqldb\1.8.0.10\hsqldb-1.8.0.10.jar;C:\Users\abc\.m2\repository\oro\oro\2.0.8\oro-2.0.8.jar;C:\Users\abc\.m2\repository\org\eclipse\jdt\core\3.1.1\core-3.1.1.jar;C:\Users\abc\.m2\repository\com\sun\mail\javax.mail\1.5.1\javax.mail-1.5.1.jar;C:\Users\abc\.m2\repository\javax\activation\activation\1.1\activation-1.1.jar 17/08/04 09:18:34 INFO zookeeper.ZooKeeper: Client environment:java.library.path=C:\Users\abc\AppData\Local\MyEclipse 2017 CI\binary\com.sun.java.jdk8.win32.x86_64_1.8.0.v112\bin;C:\windows\Sun\Java\bin;C:\windows\system32;C:\windows;C:\tibco\tibrv\8.2\bin;c:\Program Files (x86)\RSA SecurID Token Common;C:\ProgramData\Oracle\Java\javapath;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files\PuTTY\;C:\Users\abc\AppData\Local\Microsoft\WindowsApps;;. 17/08/04 09:18:34 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=C:\Users\abc\AppData\Local\Temp\1\ 17/08/04 09:18:34 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 17/08/04 09:18:34 INFO zookeeper.ZooKeeper: Client environment:os.name=Windows 10 17/08/04 09:18:34 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64 17/08/04 09:18:34 INFO zookeeper.ZooKeeper: Client environment:os.version=10.0 17/08/04 09:18:34 INFO zookeeper.ZooKeeper: Client environment:user.name=abc 17/08/04 09:18:34 INFO zookeeper.ZooKeeper: Client environment:user.home=C:\Users\abc 17/08/04 09:18:34 INFO zookeeper.ZooKeeper: Client environment:user.dir=C:\Users\abc\Workspaces\MyEclipse 2017 CI\javacode 17/08/04 09:18:34 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=en01.com:2181 sessionTimeout=90000 watcher=hconnection-0xba8d91c0x0, quorum=en01.com:2181, baseZNode=/hbase 17/08/04 09:18:34 INFO zookeeper.ClientCnxn: Opening socket connection to server en01.com/10.1.20.30:2181. Will not attempt to authenticate using SASL (unknown error) 17/08/04 09:18:34 INFO zookeeper.ClientCnxn: Socket connection established to en01.com/10.1.20.30:2181, initiating session 17/08/04 09:18:34 INFO zookeeper.ClientCnxn: Session establishment complete on server en01.com/10.1.20.30:2181, sessionid = 0x65da444b0180e25, negotiated timeout = 60000 org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions: Fri Aug 04 10:01:04 CDT 2017, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=68079: row 'tbl_abc,,99999999999999' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=dn06.com,60020,1501699598837, seqNum=0 at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:264) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:199) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:56) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200) at org.apache.hadoop.hbase.client.ClientSmallReversedScanner.next(ClientSmallReversedScanner.java:145) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1200) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1109) at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:293) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:131) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:56) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200) at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:287) at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:267) at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:139) at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:134) at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:823) at com.seagate.dars.imagedefectanalysis.dao.HBaseDao.saveFiles2FS(HBaseDao.java:175) at com.seagate.dars.imagedefectanalysis.ImageDefectAnalysis.main(ImageDefectAnalysis.java:40) Caused by: java.net.SocketTimeoutException: callTimeout=60000, callDuration=68079: row 'tbl_defect_analysis_images,,99999999999999' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=dn06.com,60020,1501699598837, seqNum=0 at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:159) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:294) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:275) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.net.ConnectTimeoutException: 10000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=dn06.com/ip:60020] at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:534) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupConnection(RpcClientImpl.java:403) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:709) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:880) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:849) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1173) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:216) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:300) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:31751) at org.apache.hadoop.hbase.client.ClientSmallScanner$SmallScannerCallable.call(ClientSmallScanner.java:176) at org.apache.hadoop.hbase.client.ClientSmallScanner$SmallScannerCallable.call(ClientSmallScanner.java:155) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126) ... 6 more
... View more
