Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

metrics failed on org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher

hi all

we try to start the ambari metrics and we get the following logs:

2019-01-02 06:35:16,406 ERROR org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher: hconnection-0x4bdeaabb-0x1680d44c1810001, quorum=master02.sys32.com:61181, baseZNode=/ams-hbase-unsecure Received unexpected KeeperException, re-throwing exception
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /ams-hbase-unsecure/table/SYSTEM.CATALOG
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
        at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(RecoverableZooKeeper.java:354)
        at org.apache.hadoop.hbase.zookeeper.ZKUtil.getData(ZKUtil.java:622)

and also

2019-01-02 06:35:18,256 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server master02.sys32.com/101.23.98.60:61181. Will not attempt to authenticate using SASL (unknown error)
2019-01-02 06:35:18,257 WARN org.apache.zookeeper.ClientCnxn: Session 0x1680d44c1810001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
2019-01-02 06:35:19,656 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server master02.sys32.com/101.23.98.60:61181. Will not attempt to authenticate using SASL (unknown error)

we also clear the contents of /var/lib/ambari-metrics-collector/hbase-tmp/zookeeper/* and restart Ambari metrics collector

but still metrics collector failed after few seconds

what else we can do in order to solve this issue ?

we have HDP version - 2.6.0


Michael-Bronson
1 ACCEPTED SOLUTION

Super Mentor

@Michael Bronson

Your current Thread query looks very similar to the other HCC thread opened by you: https://community.hortonworks.com/questions/231176/metrics-collector-ams-hbase-unsecure-received-une...

.

Can you please mark one of the HCC thread as Closed so that all the hcc users can respond to the same single thread.

View solution in original post

3 REPLIES 3

Super Mentor

@Michael Bronson

Your current Thread query looks very similar to the other HCC thread opened by you: https://community.hortonworks.com/questions/231176/metrics-collector-ams-hbase-unsecure-received-une...

.

Can you please mark one of the HCC thread as Closed so that all the hcc users can respond to the same single thread.

@Jay Kumar SenSharma , it was because when I post the first one , then we notice that first question not appears , so I post the second one

Michael-Bronson

I try also to delete this duplicate post but I cant , so I accepted your answer insted , hope this is ok

Michael-Bronson