Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Timeline Server v2.0 Reader not starting

Timeline Server v2.0 Reader not starting

New Contributor

I was using Ambari 2.6.2.2 and HDP 2.5 and HBase was not installed. Then I have upgraded Ambari to 2.7.1.0 and during the HDP upgrade Timeline Server v2.0 does not start and throws the following output:

stderr: 
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/timelinereader.py", line 119, in 
    ApplicationTimelineReader().execute()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 351, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/timelinereader.py", line 58, in start
    hbase(action='start')
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/hbase_service.py", line 80, in hbase
    createTables()
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/hbase_service.py", line 147, in createTables
    logoutput=True)
  File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 263, in action_run
    returns=self.resource.returns)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 308, in _call
    raise ExecuteTimeoutException(err_msg)
resource_management.core.exceptions.ExecuteTimeoutException: Execution of 'ambari-sudo.sh su yarn-ats -l -s /bin/bash -c 'export  PATH='"'"'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:/var/lib/ambari-agent'"'"' ; sleep 10;export HBASE_CLASSPATH_PREFIX=/usr/hdp/3.0.1.0-187/hadoop-yarn/timelineservice/*; /usr/hdp/3.0.1.0-187/hbase/bin/hbase --config /usr/hdp/3.0.1.0-187/hadoop/conf/embedded-yarn-ats-hbase org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator -Dhbase.client.retries.number=35 -create -s'' was killed due timeout after 300 seconds
resource_management.core.exceptions.ExecuteTimeoutException: Execution of 'ambari-sudo.sh su yarn-ats -l -s /bin/bash -c 'export  PATH='"'"'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:/var/lib/ambari-agent'"'"' ; sleep 10;export HBASE_CLASSPATH_PREFIX=/usr/hdp/3.0.1.0-187/hadoop-yarn/timelineservice/*; /usr/hdp/3.0.1.0-187/hbase/bin/hbase --config /usr/hdp/3.0.1.0-187/hadoop/conf/embedded-yarn-ats-hbase org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator -Dhbase.client.retries.number=35 -create -s'' was killed due timeout after 300 seconds

stdout:
INFO  [ReadOnlyZKClient-host01.xx.com:2181@0x0000f79e] zookeeper.ZooKeeper: Initiating client connection, connectString=host01.xx.com:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$13/1049642349@67016742
2018-12-08 21:07:29,083 INFO  [ReadOnlyZKClient-host01.xx.com:2181@0x0000f79e-SendThread(host01.xx.com:2181)] zookeeper.ClientCnxn: Opening socket connection to server host01.xx.com/10.0.3.15:2181. Will not attempt to authenticate using SASL (unknown error)
2018-12-08 21:07:29,089 INFO  [ReadOnlyZKClient-host01.xx.com:2181@0x0000f79e-SendThread(host01.xx.com:2181)] zookeeper.ClientCnxn: Socket connection established, initiating session, client: /10.0.3.15:38691, server: host01.xx.com/10.0.3.15:2181
2018-12-08 21:07:29,103 INFO  [ReadOnlyZKClient-host01.xx.com:2181@0x0000f79e-SendThread(host01.xx.com:2181)] zookeeper.ClientCnxn: Session establishment complete on server host01.xx.com/10.0.3.15:2181, sessionid = 0x1679012b5940157, negotiated timeout = 60000
2018-12-08 21:07:33,523 INFO  [main] client.RpcRetryingCallerImpl: Call exception, tries=6, retries=36, started=4153 ms ago, cancelled=false, msg=org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /atsv2-hbase-unsecure/meta-region-server, details=row 'prod.timelineservice.entity' on table 'hbase:meta' at null
2018-12-08 21:07:37,535 INFO  [main] client.RpcRetryingCallerImpl: Call exception, tries=7, retries=36, started=8166 ms ago, cancelled=false, msg=org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /atsv2-hbase-unsecure/meta-region-server, details=row 'prod.timelineservice.entity' on table 'hbase:meta' at null
2018-12-08 21:07:47,605 INFO  [main] client.RpcRetryingCallerImpl: Call exception, tries=8, retries=36, started=18236 ms ago, cancelled=false, msg=org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /atsv2-hbase-unsecure/meta-region-server, details=row 'prod.timelineservice.entity' on table 'hbase:meta' at null

1 REPLY 1

Re: Timeline Server v2.0 Reader not starting

Expert Contributor

Hi @fraklin J,

It's seems like Hbase client need to installed in the host.

Could you please check if the host where the TimelineServer V2.0 Reader is installed also has the "HBase Client" installed or not?

If not then please install the "HBase client" on that host and give a try.

Please login and accept the answer if you find it helpful :)

Don't have an account?
Coming from Hortonworks? Activate your account here