Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Zookeeper server is not coming up

Contributor

Hello Experts,

 

One of the zookeeper server out of three has gone down and after applying various workaround it is still not coming up. We have three in total including one as the zookeeper leader and other two are followers.

 

Error:

INFO org.apache.zookeeper.server.persistence.FileSnap: Reading snapshot /var/lib/zookeeper/version-2/snapshot.6801207d537
2022-11-05 13:54:57,287 ERROR org.apache.zookeeper.server.quorum.QuorumPeer: Unable to load database on disk
java.io.EOFException

 

Tried restarting multiple times all zookeeper servers and also just restarting leader and then followers but issue still persist.

 

Also moved the snapshot to tmp by running command

mv /var//lib/zookeeper/version-2/snapshot.6801207d537 /tmp

and then restarted but not coming up.

 

Please advise what else can be done in this scenario.

3 REPLIES 3

New Contributor

Contributor

Yes I had already applied this workaround but it didn't work. 

Cloudera Employee

Hi Hanzala,

Thank you for raising concern in community.

If the snapshot is moved then the info will not the show the below error::
--------------

 Reading snapshot /var/lib/zookeeper/version-2/snapshot.6801207d537

 

There might some disk issue which cause the snapshot to be not end the file properly

from the problematic zookeeper

1.move the ZK's Data Directory to backup Directory
2. Copy the transition logs and version-2/* from any running zookeeper to problematic zookeeper node
3. Then try starting problematic ZK node again

And please update us the new log 
Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.