org.apache.hadoop.hbase.ipc.ServerNotRunningYetException

amit_hadoop — Wed, 29 Sep 2021 09:51:58 GMT

Hi,

I have installed Hortonworks Sandbox HDP3.0

I leave all services to start there is an issue with starting Timeline Service V2.0 Reader

2021-09-29 05:47:58,587 INFO  [main] client.RpcRetryingCallerImpl: Call exception, tries=6, retries=36, started=5385 ms ago, cancelled=false, msg=org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server sandbox-hdp.hortonworks.com,17020,1632894462010 is not running yet
	at org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1487)
	at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2443)
	at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
, details=row 'prod.timelineservice.entity' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=sandbox-hdp.hortonworks.com,17020,1628801419880, seqNum=-1

I have started HDFS and HBASE both show as running. Initially HBASE was in maintainance mode which I have removed and did Restart.

Thanks

Re: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException

smdas — Thu, 21 Oct 2021 07:16:10 GMT

Hello @amit_hadoop

Thanks for using Cloudera Community. Based on the Post, You installed HDP Sandbox 3.0 & observed Timeline Service v2 Reader reporting [1].

If you observe the Port mentioned, It says "17020" & HBase RegionServer runs with "16020" Port by default. YARN Timeline Service uses an Embedded HBase Service, which isn't equivalent to HBase Service as we may see in the Ambari UI. Kindly let us know the Outcome of the below Steps:

Whether Restarting YARN Timeline Service (Which restarts the Embedded HBase Service) helps,
Within the YARN Logs, We should have an Embedded HBase Logs for HMaster & RegionServer. Reviewing the Logs may offer additional clue into the reasoning for Embedded HBase concerns,
Review [2] for similar concerns wherein the ZNode was Cleared & Service Restarted,
Review [3] for Documentation on Deleting the ZNode & Data. Restarting the YARN Timeline Service should initialise the HBase Setup afresh.

Regards, Smarak

[1] Error Trace:

2021-09-29 05:47:58,587 INFO [main] client.RpcRetryingCallerImpl: Call exception, tries=6, retries=36, started=5385 ms ago, cancelled=false, msg=org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server sandbox-hdp.hortonworks.com,17020,1632894462010 is not running yet details=row 'prod.timelineservice.entity' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=sandbox-hdp.hortonworks.com,17020,1628801419880, seqNum=-1

[2] https://community.cloudera.com/t5/Support-Questions/Yarn-Timeline-Service-V2-not-starting/td-p/281368

[3] https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.4/data-operating-system/content/remove_ats_hbase_before_switching_between_clusters.html

question org.apache.hadoop.hbase.ipc.ServerNotRunningYetException in Support Questions

org.apache.hadoop.hbase.ipc.ServerNotRunningYetException

Re: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException