Member since
01-22-2016
26
Posts
10
Kudos Received
2
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 3273 | 12-09-2016 08:51 AM | |
| 3018 | 09-14-2016 07:23 AM |
09-14-2016
07:23 AM
Hello All, Thanks all for your help, I managed to resolve the issue by doing the following: 1) I realized that in the ambari-agent logs I have this error message {'msg':'Unable to read structured output from /var/lib/ambari-agent/data/structured-out-status.json'} 2) For resolving this first stop ambari agent 3) move the move /var/lib/ambari-agent/data/structured-out-status.json to /tmp. 4) Restart the ambari-agent. 5) Everything is green again. Actually I followed a post mentioned here: https://community.hortonworks.com/questions/16953/unable-to-start-hdp-serrvices-from-ambari.html Don't know the exact reason behind this, but i was able to fix the issue. Thanks again for your help.
... View more
12-14-2016
02:01 AM
@Hammad Ali Seems like i was able to fix the installation issue, however on starting OpenTSDB, the following error was thrown. Any ideas ? ------------------------ 2016-12-14 00:51:16,030 INFO [AsyncHBase I/O Worker #3] HBaseClient: Channel [id: 0x0d57d67b, /10.0.2.15:37370 😆 /10.0.2.15:16020] is disconnecting: [id: 0x0d57d67b, /10.0.2.15:37370 😆 /10.0.2.15:16020] DISCONNECTED
2016-12-14 00:51:16,032 WARN [AsyncHBase Timer HBaseClient #1-EventThread] HBaseClient: Probe Exists(table="hbase:meta", key=[116, 115, 100, 98, 44, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 58, 65, 115, 121, 110, 99, 72, 66, 97, 115, 101, 126, 112, 114, 111, 98, 101, 126, 60, 59, 95, 60, 44, 58, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 58, 65, 115, 121, 110, 99, 72, 66, 97, 115, 101, 126, 112, 114, 111, 98, 101, 126, 60, 59, 95, 60], family=null, qualifiers=null, attempt=0, region=RegionInfo(table="hbase:meta", region_name="hbase:meta,,1", stop_key="")) failed
org.hbase.async.NonRecoverableException: Too many attempts: Exists(table="hbase:meta", key=[116, 115, 100, 98, 44, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 58, 65, 115, 121, 110, 99, 72, 66, 97, 115, 101, 126, 112, 114, 111, 98, 101, 126, 60, 59, 95, 60, 44, 58, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 58, 65, 115, 121, 110, 99, 72, 66, 97, 115, 101, 126, 112, 114, 111, 98, 101, 126, 60, 59, 95, 60], family=null, qualifiers=null, attempt=11, region=RegionInfo(table="hbase:meta", region_name="hbase:meta,,1", stop_key=""))
at org.hbase.async.HBaseClient.tooManyAttempts(HBaseClient.java:2056) [asynchbase-1.7.2.jar:na]
at org.hbase.async.HBaseClient.sendRpcToRegion(HBaseClient.java:1920) [asynchbase-1.7.2.jar:na]
at org.hbase.async.HBaseClient$1RetryRpc.call(HBaseClient.java:1944) [asynchbase-1.7.2.jar:na]
at org.hbase.async.HBaseClient$1RetryRpc.call(HBaseClient.java:1927) [asynchbase-1.7.2.jar:na]
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1278) [async-1.4.0.jar:na]
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1257) [async-1.4.0.jar:na]
at com.stumbleupon.async.Deferred.callback(Deferred.java:1005) [async-1.4.0.jar:na]
at org.hbase.async.HBaseClient$ZKClient$ZKCallback.processResult(HBaseClient.java:3645) [asynchbase-1.7.2.jar:na]
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:558) [zookeeper-3.4.5.jar:3.4.5-1392090]
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495) [zookeeper-3.4.5.jar:3.4.5-1392090]
2016-12-14 00:51:16,033 INFO [AsyncHBase Timer HBaseClient #1-EventThread] HBaseClient: Need to find the .META. region
2016-12-14 00:51:16,037 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ZooKeeper: Session: 0x158fa6cce65008d closed
2016-12-14 00:51:16,037 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ZooKeeper: Initiating client connection, connectString=sandbox.hortonworks.com:2181 sessionTimeout=5000 watcher=org.hbase.async.HBaseClient$ZKClient@6e232ae3
2016-12-14 00:51:16,038 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Opening socket connection to server sandbox.hortonworks.com/10.0.2.15:2181. Will not attempt to authenticate using SASL (unknown error)
2016-12-14 00:51:16,040 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Socket connection established to sandbox.hortonworks.com/10.0.2.15:2181, initiating session
2016-12-14 00:51:16,042 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Session establishment complete on server sandbox.hortonworks.com/10.0.2.15:2181, sessionid = 0x158fa6cce65008e, negotiated timeout = 5000
2016-12-14 00:51:16,043 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ClientCnxn: EventThread shut down
2016-12-14 00:51:16,048 INFO [AsyncHBase Timer HBaseClient #1-EventThread] HBaseClient: Connecting to .META. region @ 10.0.2.15:16020
2016-12-14 00:51:16,051 INFO [AsyncHBase I/O Worker #4] HBaseClient: Channel [id: 0xf2cf1dfe, /10.0.2.15:37372 😆 /10.0.2.15:16020] is disconnecting: [id: 0xf2cf1dfe, /10.0.2.15:37372 😆 /10.0.2.15:16020] DISCONNECTED
2016-12-14 00:51:16,052 INFO [AsyncHBase Timer HBaseClient #1-EventThread] HBaseClient: Need to find the .META. region
2016-12-14 00:51:16,055 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ZooKeeper: Session: 0x158fa6cce65008e closed
2016-12-14 00:51:16,055 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ZooKeeper: Initiating client connection, connectString=sandbox.hortonworks.com:2181 sessionTimeout=5000 watcher=org.hbase.async.HBaseClient$ZKClient@6e232ae3
2016-12-14 00:51:16,056 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Opening socket connection to server sandbox.hortonworks.com/10.0.2.15:2181. Will not attempt to authenticate using SASL (unknown error)
2016-12-14 00:51:16,058 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Socket connection established to sandbox.hortonworks.com/10.0.2.15:2181, initiating session
2016-12-14 00:51:16,061 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Session establishment complete on server sandbox.hortonworks.com/10.0.2.15:2181, sessionid = 0x158fa6cce65008f, negotiated timeout = 5000
2016-12-14 00:51:16,061 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ClientCnxn: EventThread shut down
2016-12-14 00:51:16,067 INFO [AsyncHBase Timer HBaseClient #1-EventThread] HBaseClient: Connecting to .META. region @ 10.0.2.15:16020
2016-12-14 00:51:16,069 INFO [AsyncHBase I/O Worker #1] HBaseClient: Channel [id: 0x24f85c35, /10.0.2.15:37374 😆 /10.0.2.15:16020] is disconnecting: [id: 0x24f85c35, /10.0.2.15:37374 😆 /10.0.2.15:16020] DISCONNECTED
2016-12-14 00:51:16,070 INFO [AsyncHBase Timer HBaseClient #1-EventThread] HBaseClient: Need to find the .META. region
2016-12-14 00:51:16,074 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ZooKeeper: Session: 0x158fa6cce65008f closed
2016-12-14 00:51:16,074 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ZooKeeper: Initiating client connection, connectString=sandbox.hortonworks.com:2181 sessionTimeout=5000 watcher=org.hbase.async.HBaseClient$ZKClient@6e232ae3
2016-12-14 00:51:16,075 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Opening socket connection to server sandbox.hortonworks.com/10.0.2.15:2181. Will not attempt to authenticate using SASL (unknown error)
2016-12-14 00:51:16,078 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Socket connection established to sandbox.hortonworks.com/10.0.2.15:2181, initiating session
2016-12-14 00:51:16,080 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ClientCnxn: EventThread shut down
2016-12-14 00:51:16,084 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Session establishment complete on server sandbox.hortonworks.com/10.0.2.15:2181, sessionid = 0x158fa6cce650090, negotiated timeout = 5000
2016-12-14 00:51:16,087 INFO [AsyncHBase Timer HBaseClient #1-EventThread] HBaseClient: Connecting to .META. region @ 10.0.2.15:16020
2016-12-14 00:51:16,089 INFO [AsyncHBase I/O Worker #2] HBaseClient: Channel [id: 0xd63aca14, /10.0.2.15:37376 😆 /10.0.2.15:16020] is disconnecting: [id: 0xd63aca14, /10.0.2.15:37376 😆 /10.0.2.15:16020] DISCONNECTED
2016-12-14 00:51:16,090 INFO [AsyncHBase Timer HBaseClient #1-EventThread] HBaseClient: Need to find the .META. region
2016-12-14 00:51:16,092 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ZooKeeper: Session: 0x158fa6cce650090 closed
2016-12-14 00:51:16,092 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ZooKeeper: Initiating client connection, connectString=sandbox.hortonworks.com:2181 sessionTimeout=5000 watcher=org.hbase.async.HBaseClient$ZKClient@6e232ae3
2016-12-14 00:51:16,094 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Opening socket connection to server sandbox.hortonworks.com/10.0.2.15:2181. Will not attempt to authenticate using SASL (unknown error)
2016-12-14 00:51:16,095 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Socket connection established to sandbox.hortonworks.com/10.0.2.15:2181, initiating session
2016-12-14 00:51:16,098 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Session establishment complete on server sandbox.hortonworks.com/10.0.2.15:2181, sessionid = 0x158fa6cce650091, negotiated timeout = 5000
2016-12-14 00:51:16,098 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ClientCnxn: EventThread shut down
2016-12-14 00:51:16,104 INFO [AsyncHBase Timer HBaseClient #1-EventThread] HBaseClient: Connecting to .META. region @ 10.0.2.15:16020
2016-12-14 00:51:16,106 INFO [AsyncHBase I/O Worker #3] HBaseClient: Channel [id: 0x24387fa0, /10.0.2.15:37378 😆 /10.0.2.15:16020] is disconnecting: [id: 0x24387fa0, /10.0.2.15:37378 😆 /10.0.2.15:16020] DISCONNECTED
2016-12-14 00:51:16,107 INFO [AsyncHBase Timer HBaseClient #1-EventThread] HBaseClient: Need to find the .META. region
2016-12-14 00:51:16,110 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ZooKeeper: Session: 0x158fa6cce650091 closed
2016-12-14 00:51:16,110 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ZooKeeper: Initiating client connection, connectString=sandbox.hortonworks.com:2181 sessionTimeout=5000 watcher=org.hbase.async.HBaseClient$ZKClient@6e232ae3
2016-12-14 00:51:16,110 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Opening socket connection to server sandbox.hortonworks.com/10.0.2.15:2181. Will not attempt to authenticate using SASL (unknown error)
2016-12-14 00:51:16,112 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Socket connection established to sandbox.hortonworks.com/10.0.2.15:2181, initiating session
2016-12-14 00:51:16,115 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Session establishment complete on server sandbox.hortonworks.com/10.0.2.15:2181, sessionid = 0x158fa6cce650092, negotiated timeout = 5000
2016-12-14 00:51:16,116 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ClientCnxn: EventThread shut down
2016-12-14 00:51:16,122 INFO [AsyncHBase Timer HBaseClient #1-EventThread] HBaseClient: Connecting to .META. region @ 10.0.2.15:16020
2016-12-14 00:51:16,126 INFO [AsyncHBase I/O Worker #4] HBaseClient: Channel [id: 0x8d3a08ca, /10.0.2.15:37380 😆 /10.0.2.15:16020] is disconnecting: [id: 0x8d3a08ca, /10.0.2.15:37380 😆 /10.0.2.15:16020] DISCONNECTED
2016-12-14 00:51:16,127 INFO [AsyncHBase Timer HBaseClient #1-EventThread] HBaseClient: Need to find the .META. region
2016-12-14 00:51:16,130 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ZooKeeper: Session: 0x158fa6cce650092 closed
2016-12-14 00:51:16,130 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ZooKeeper: Initiating client connection, connectString=sandbox.hortonworks.com:2181 sessionTimeout=5000 watcher=org.hbase.async.HBaseClient$ZKClient@6e232ae3
2016-12-14 00:51:16,131 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Opening socket connection to server sandbox.hortonworks.com/10.0.2.15:2181. Will not attempt to authenticate using SASL (unknown error)
2016-12-14 00:51:16,132 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Socket connection established to sandbox.hortonworks.com/10.0.2.15:2181, initiating session
2016-12-14 00:51:16,135 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Session establishment complete on server sandbox.hortonworks.com/10.0.2.15:2181, sessionid = 0x158fa6cce650093, negotiated timeout = 5000
2016-12-14 00:51:16,136 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ClientCnxn: EventThread shut down
2016-12-14 00:51:16,143 INFO [AsyncHBase Timer HBaseClient #1-EventThread] HBaseClient: Connecting to .META. region @ 10.0.2.15:16020
2016-12-14 00:51:16,146 INFO [AsyncHBase I/O Worker #1] HBaseClient: Channel [id: 0x43a1016f, /10.0.2.15:37382 😆 /10.0.2.15:16020] is disconnecting: [id: 0x43a1016f, /10.0.2.15:37382 😆 /10.0.2.15:16020] DISCONNECTED
2016-12-14 00:51:16,147 INFO [AsyncHBase Timer HBaseClient #1-EventThread] HBaseClient: Need to find the .META. region
2016-12-14 00:51:16,150 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ZooKeeper: Session: 0x158fa6cce650093 closed
2016-12-14 00:51:16,150 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ZooKeeper: Initiating client connection, connectString=sandbox.hortonworks.com:2181 sessionTimeout=5000 watcher=org.hbase.async.HBaseClient$ZKClient@6e232ae3
2016-12-14 00:51:16,151 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Opening socket connection to server sandbox.hortonworks.com/10.0.2.15:2181. Will not attempt to authenticate using SASL (unknown error)
2016-12-14 00:51:16,153 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Socket connection established to sandbox.hortonworks.com/10.0.2.15:2181, initiating session
2016-12-14 00:51:16,155 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ClientCnxn: EventThread shut down
2016-12-14 00:51:16,160 INFO [AsyncHBase Timer HBaseClient #1-SendThread(sandbox.hortonworks.com:2181)] ClientCnxn: Session establishment complete on server sandbox.hortonworks.com/10.0.2.15:2181, sessionid = 0x158fa6cce650094, negotiated timeout = 5000
2016-12-14 00:51:16,163 INFO [AsyncHBase Timer HBaseClient #1-EventThread] HBaseClient: Connecting to .META. region @ 10.0.2.15:16020
2016-12-14 00:51:16,167 INFO [AsyncHBase I/O Worker #2] HBaseClient: Channel [id: 0x9b61659f, /10.0.2.15:37384 😆 /10.0.2.15:16020] is disconnecting: [id: 0x9b61659f, /10.0.2.15:37384 😆 /10.0.2.15:16020] DISCONNECTED
2016-12-14 00:51:16,174 INFO [main] TSDB: Flushing compaction queue
2016-12-14 00:51:16,185 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ZooKeeper: Session: 0x158fa6cce650094 closed
2016-12-14 00:51:16,185 INFO [AsyncHBase Timer HBaseClient #1-EventThread] ClientCnxn: EventThread shut down
2016-12-14 00:51:16,191 INFO [main] TSDB: Completed shutting down the TSDB
Exception in thread "main" java.lang.RuntimeException: Initialization failed
at net.opentsdb.tools.TSDMain.main(TSDMain.java:196)
Caused by: com.stumbleupon.async.DeferredGroupException: At least one of the Deferreds failed, first exception:
at com.stumbleupon.async.DeferredGroup.done(DeferredGroup.java:169)
at com.stumbleupon.async.DeferredGroup.recordCompletion(DeferredGroup.java:142)
at com.stumbleupon.async.DeferredGroup.access$000(DeferredGroup.java:36)
at com.stumbleupon.async.DeferredGroup$1Notify.call(DeferredGroup.java:82)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1278)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1257)
at com.stumbleupon.async.Deferred.access$300(Deferred.java:430)
at com.stumbleupon.async.Deferred$Continue.call(Deferred.java:1366)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1278)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1257)
at com.stumbleupon.async.Deferred.handleContinuation(Deferred.java:1313)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1284)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1257)
at com.stumbleupon.async.Deferred.callback(Deferred.java:1005)
at org.hbase.async.HBaseRpc.callback(HBaseRpc.java:712)
at org.hbase.async.HBaseClient.tooManyAttempts(HBaseClient.java:2058)
at org.hbase.async.HBaseClient.sendRpcToRegion(HBaseClient.java:1920)
at org.hbase.async.HBaseClient$1RetryRpc.call(HBaseClient.java:1944)
at org.hbase.async.HBaseClient$1RetryRpc.call(HBaseClient.java:1927)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1278)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1257)
at com.stumbleupon.async.Deferred.callback(Deferred.java:1005)
at org.hbase.async.HBaseClient$ZKClient$ZKCallback.processResult(HBaseClient.java:3645)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:558)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
Caused by: org.hbase.async.NonRecoverableException: Too many attempts: HBaseRpc(method=getClosestRowBefore, table="hbase:meta", key=[116, 115, 100, 98, 45, 117, 105, 100, 44, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 58, 65, 115, 121, 110, 99, 72, 66, 97, 115, 101, 126, 112, 114, 111, 98, 101, 126, 60, 59, 95, 60, 44, 58], region=RegionInfo(table="hbase:meta", region_name="hbase:meta,,1", stop_key=""), attempt=11, timeout=0, hasTimedout=false)
at org.hbase.async.HBaseClient.tooManyAttempts(HBaseClient.java:2056)
... 9 more
... View more
11-15-2016
03:00 AM
@Hammad Ali, The problem has been solved .And I have not install ranger plugin from ambari. Thank for you answer.Thank you.
... View more