Created 12-09-2018 07:19 PM
I was using Ambari 2.6.2.2 and HDP 2.5 and HBase was not installed. Then I have upgraded Ambari to 2.7.1.0 and during the HDP upgrade Timeline Server v2.0 does not start and throws the following output:
stderr: Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/timelinereader.py", line 119, in ApplicationTimelineReader().execute() File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 351, in execute method(env) File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/timelinereader.py", line 58, in start hbase(action='start') File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/hbase_service.py", line 80, in hbase createTables() File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/hbase_service.py", line 147, in createTables logoutput=True) File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__ self.env.run() File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 263, in action_run returns=self.resource.returns) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner result = function(command, **kwargs) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 308, in _call raise ExecuteTimeoutException(err_msg) resource_management.core.exceptions.ExecuteTimeoutException: Execution of 'ambari-sudo.sh su yarn-ats -l -s /bin/bash -c 'export PATH='"'"'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:/var/lib/ambari-agent'"'"' ; sleep 10;export HBASE_CLASSPATH_PREFIX=/usr/hdp/3.0.1.0-187/hadoop-yarn/timelineservice/*; /usr/hdp/3.0.1.0-187/hbase/bin/hbase --config /usr/hdp/3.0.1.0-187/hadoop/conf/embedded-yarn-ats-hbase org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator -Dhbase.client.retries.number=35 -create -s'' was killed due timeout after 300 seconds resource_management.core.exceptions.ExecuteTimeoutException: Execution of 'ambari-sudo.sh su yarn-ats -l -s /bin/bash -c 'export PATH='"'"'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:/var/lib/ambari-agent'"'"' ; sleep 10;export HBASE_CLASSPATH_PREFIX=/usr/hdp/3.0.1.0-187/hadoop-yarn/timelineservice/*; /usr/hdp/3.0.1.0-187/hbase/bin/hbase --config /usr/hdp/3.0.1.0-187/hadoop/conf/embedded-yarn-ats-hbase org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator -Dhbase.client.retries.number=35 -create -s'' was killed due timeout after 300 seconds stdout: INFO [ReadOnlyZKClient-host01.xx.com:2181@0x0000f79e] zookeeper.ZooKeeper: Initiating client connection, connectString=host01.xx.com:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$13/1049642349@67016742 2018-12-08 21:07:29,083 INFO [ReadOnlyZKClient-host01.xx.com:2181@0x0000f79e-SendThread(host01.xx.com:2181)] zookeeper.ClientCnxn: Opening socket connection to server host01.xx.com/10.0.3.15:2181. Will not attempt to authenticate using SASL (unknown error) 2018-12-08 21:07:29,089 INFO [ReadOnlyZKClient-host01.xx.com:2181@0x0000f79e-SendThread(host01.xx.com:2181)] zookeeper.ClientCnxn: Socket connection established, initiating session, client: /10.0.3.15:38691, server: host01.xx.com/10.0.3.15:2181 2018-12-08 21:07:29,103 INFO [ReadOnlyZKClient-host01.xx.com:2181@0x0000f79e-SendThread(host01.xx.com:2181)] zookeeper.ClientCnxn: Session establishment complete on server host01.xx.com/10.0.3.15:2181, sessionid = 0x1679012b5940157, negotiated timeout = 60000 2018-12-08 21:07:33,523 INFO [main] client.RpcRetryingCallerImpl: Call exception, tries=6, retries=36, started=4153 ms ago, cancelled=false, msg=org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /atsv2-hbase-unsecure/meta-region-server, details=row 'prod.timelineservice.entity' on table 'hbase:meta' at null 2018-12-08 21:07:37,535 INFO [main] client.RpcRetryingCallerImpl: Call exception, tries=7, retries=36, started=8166 ms ago, cancelled=false, msg=org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /atsv2-hbase-unsecure/meta-region-server, details=row 'prod.timelineservice.entity' on table 'hbase:meta' at null 2018-12-08 21:07:47,605 INFO [main] client.RpcRetryingCallerImpl: Call exception, tries=8, retries=36, started=18236 ms ago, cancelled=false, msg=org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /atsv2-hbase-unsecure/meta-region-server, details=row 'prod.timelineservice.entity' on table 'hbase:meta' at null
Created 12-10-2018 05:54 AM
Hi @fraklin J,
It's seems like Hbase client need to installed in the host.
Could you please check if the host where the TimelineServer V2.0 Reader is installed also has the "HBase Client" installed or not?
If not then please install the "HBase client" on that host and give a try.
Please login and accept the answer if you find it helpful 🙂