Support Questions

Find answers, ask questions, and share your expertise

habse region server not communicatiin Hmaster

avatar
Explorer

2018-04-25 13:56:47,626 WARN [regionserver//hostname:16020] regionserver.HRegionServer: error telling master we are up com.google.protobuf.ServiceException: java.io.IOException: Call to hostname/:16000 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=0, waitTime=10002, operationTimeout=10000 expired. at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:228) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:292) at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:10859) at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2437) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:955) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Call to hostname/:16000 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=0, waitTime=10002, operationTimeout=10000 expired. at org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1263) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1231) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:218) ... 5 more Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=0, waitTime=10002, operationTimeout=10000 expired. at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1205) ... 6 more 2018-04-25 13:56:47,628 WARN [regionserver//hostname:16020] regionserver.HRegionServer: reportForDuty failed; sleeping and then retrying. 2018-04-25 13:56:50,628 INFO [regionserver//hostname:16020] regionserver.HRegionServer: reportForDuty to master=hostname,16000,1524567321593 with port=16020, startcode=1524664595268 2018-04-25 13:57:00,629 WARN [regionserver//hostname:16020] regionserver.HRegionServer: error telling master we are up com.google.protobuf.ServiceException: java.io.IOException: Call to hostname/:16000 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=1, waitTime=10001, operationTimeout=10000 expired. at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:228) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:292) at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:10859) at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2437) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:955) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Call to hostname/:16000 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=1, waitTime=10001, operationTimeout=10000 expired. at org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1263) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1231) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:218) ... 5 more Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=1, waitTime=10001, operationTimeout=10000 expired. at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1205) ... 6 more 2018-04-25 13:57:00,629 WARN [regionserver//hostname:16020] regionserver.HRegionServer: reportForDuty failed; sleeping and then retrying. 2018-04-25 13:57:03,630 INFO [regionserver//hostname:16020] regionserver.HRegionServer: reportForDuty to master=hostname,16000,1524567321593 with port=16020, startcode=1524664595268 2018-04-25 13:57:13,632 WARN [regionserver//hostname:16020] regionserver.HRegionServer: error telling master we are up com.google.protobuf.ServiceException: java.io.IOException: Call to hostname/:16000 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=2, waitTime=10001, operationTimeout=10000 expired. at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:228) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:292) at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:10859) at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2437) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:955) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Call to hostname/:16000 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=2, waitTime=10001, operationTimeout=10000 expired. at org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1263) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1231) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:218) ... 5 more Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=2, waitTime=10001, operationTimeout=10000 expired. at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1205) ... 6 more 2018-04-25 13:57:13,632 WARN [regionserver//hostname:16020] regionserver.HRegionServer: reportForDuty failed; sleeping and then retrying. 2018-04-25 13:57:16,635 INFO [regionserver//hostname:16020] regionserver.HRegionServer: reportForDuty to master=hostname,16000,1524567321593 with port=16020, startcode=1524664595268 2018-04-25 13:57:26,636 WARN [regionserver//hostname:16020] regionserver.HRegionServer: error telling master we are up com.google.protobuf.ServiceException: java.io.IOException: Call to hostname/:16000 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=3, waitTime=10001, operationTimeout=10000 expired. at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:228) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:292) at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:10859) at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2437) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:955) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Call to hostname/:16000 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=3, waitTime=10001, operationTimeout=10000 expired. at org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1263) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1231) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:218) ... 5 more Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=3, waitTime=10001, operationTimeout=10000 expired. at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1205) ... 6 more

2 REPLIES 2

avatar
Expert Contributor

Hi @raj pati,

Ensure that Hbase master is up and running.

Please check hbase master logs for error messages.

-Shubham

avatar

@raj pati I believe communication issue was resolved after removing the external jars from hbase libs.