Support Questions
Find answers, ask questions, and share your expertise

ZOOKEEPER_CANARY_HEALTH has become bad: Canary test failed to establish a connection or a client session to the ZooKeeper service.

ZOOKEEPER_CANARY_HEALTH has become bad: Canary test failed to establish a connection or a client session to the ZooKeeper service.

Explorer

 

Health Test Name 	                 Event Code 	        Severity ZOOKEEPER_CANARY_HEALTH 	Service health test bad 	Critical 	

Content: The health test result for ZOOKEEPER_CANARY_HEALTH has become bad: Canary test failed to establish a connection or a client session to the ZooKeeper service.

 

 

 

 

We are getting like this in Zookeeper health history

ZooKeeper Canary Good
1 Still Concerning
11:01:51 PM
 	
ZooKeeper Canary Bad
11:01:01 PM
 	
ZooKeeper Canary Good
1 Still Concerning
Nov 28 7:38:30 PM
 	
ZooKeeper Canary Bad
Nov 28 7:37:34 PM
 	
ZooKeeper Canary Good
1 Still Concerning
Nov 28 10:31:06 AM
 	
ZooKeeper Canary Bad
Nov 28 10:30:21 AM
 	
ZooKeeper Canary Good
1 Still Concerning
Nov 27 11:57:47 AM

 

 

 

 

Zookeeper service helth history

ZooKeeper Canary
Canary test failed to delete a znode.

 

 

 

 

Log File from  Zookeeper Follower

11:01:17.569 PM	WARN	Learner	
Exception when following the leader
java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:171)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
	at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:85)
	at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99)
	at org.apache.zookeeper.server.quorum.Learner.readPacket(Learner.java:153)
	at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:86)
	at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:865)
11:01:17.572 PM	INFO	Learner	
shutdown called
java.lang.Exception: shutdown Follower
	at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:167)
	at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:869)
11:01:17.572 PM	INFO	FollowerZooKeeperServer	
Shutting down
11:01:17.572 PM	INFO	ZooKeeperServer	
shutting down
11:01:17.572 PM	INFO	FollowerRequestProcessor	
Shutting down
11:01:17.572 PM	INFO	CommitProcessor	
Shutting down
11:01:17.572 PM	INFO	FinalRequestProcessor	
shutdown of request processor complete
11:01:17.572 PM	INFO	SyncRequestProcessor	
Shutting down
11:01:17.573 PM	INFO	FollowerRequestProcessor	
FollowerRequestProcessor exited loop!
11:00:48.335 PM	INFO	NIOServerCnxnFactory	
Accepted socket connection from /172.27.129.30:45000
11:01:17.646 PM	WARN	NIOServerCnxn	
Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
11:01:17.646 PM	INFO	NIOServerCnxn	
Closed socket connection for client /172.27.129.30:45000 (no session established for client)
11:01:17.646 PM	INFO	CommitProcessor	
CommitProcessor exited loop!
11:01:17.648 PM	INFO	SyncRequestProcessor	
SyncRequestProcessor exited!
11:01:17.648 PM	INFO	QuorumPeer	
LOOKING

 

 

 

Zookeeper Canery

Bad : Canary test failed to establish a connection or a client session to the ZooKeeper service.

 

 

 

values present using:

 ZooKeeper Canary Connection Timeout,:10sec
 ZooKeeper Canary Session Timeout,:30sec
 ZooKeeper Canary Operation Timeout :30sec

 

 

 

Don't have an account?