Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

rolling upgrade HDP 2.2.4 to 2.3.4 issue on Zookeeper component

Solved Go to solution

rolling upgrade HDP 2.2.4 to 2.3.4 issue on Zookeeper component

Guru

Hi,

after covering prereq. for a rolling upgrade of HDP 2.2.4 to 2.3.4 (using Ambari 2.2.1.1) and starting the procedure, the first component, Zookeeper, fails with below error ("##zookeeperserver1##" is just a placeholder for real server name):

...

2016-06-03 13:42:07,725 - Will retry 1 time(s), caught exception: Failed to establish zookeeper quorum. Sleeping for 2 sec(s)
2016-06-03 13:42:09,727 - call['echo 'create /id1facad4c_date410316 mydata' | /usr/hdp/current/zookeeper-server/bin/zkCli.sh -server ##zookeeperserver1##:2181'] {'logoutput': True, 'user': 'zookeeper', 'quiet': False}
Connecting to ##zookeeperserver1##:2181
Welcome to ZooKeeper!
JLine support is enabled

WATCHER::

WatchedEvent state:AuthFailed type:None path:null
[zk: ##zookeeperserver1##:2181(CONNECTING) 0] create /id1facad4c_date410316 mydata

WATCHER::

WatchedEvent state:Disconnected type:None path:null
Exception in thread "main" org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /id1facad4c_date410316
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
	at org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:698)
	at org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:591)
	at org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:363)
	at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:323)
	at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:282)
2016-06-03 13:42:10,174 - call returned (1, 'Connecting to ##zookeeperserver1##:2181\nWelcome to ZooKeeper!\nJLine support is enabled\n\nWATCHER::\n\nWatchedEvent state:AuthFailed type:None path:null\n[zk: ##zookeeperserver1##:2181(CONNECTING) 0] create /id1facad4c_date410316 mydata\n\nWATCHER::\n\nWatchedEvent state:Disconnected type:None path:null\nException in thread "main" org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /id1facad4c_date410316\n\tat org.apache.zookeeper.KeeperException.create(KeeperException.java:99)\n\tat org.apache.zookeeper.KeeperException.create(KeeperException.java:51)\n\tat org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)\n\tat org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:698)\n\tat org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:591)\n\tat org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:363)\n\tat org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:323)\n\tat org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:282)')

What to do to be able to proceed with the upgrade ?!?!

Thanks, Gerd

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: rolling upgrade HDP 2.2.4 to 2.3.4 issue on Zookeeper component

Guru

Hi @Jitendra Yadav , thanks for getting back to me.

In the meantime the issue got solved recently through a support call.....

It looks like the upgrade process ran into a timeout while talking to Zookeeper. To proceed with the upgrade, the solution was to do "Pause Upgrade", go to Ambari Dashboard, check state of Zookeeper (which was fine, all up and running), then click "Resume Upgrade". After that the problem was gone and upgrade procedure continued.

Best regards, Gerd

View solution in original post

2 REPLIES 2
Highlighted

Re: rolling upgrade HDP 2.2.4 to 2.3.4 issue on Zookeeper component

@Gerd Koenig

It seems like the authentication was failed with zookeeper. Through which user you are perform the upgrade and is this a secure cluster?

WatchedEvent state:AuthFailed type:None path:null
Highlighted

Re: rolling upgrade HDP 2.2.4 to 2.3.4 issue on Zookeeper component

Guru

Hi @Jitendra Yadav , thanks for getting back to me.

In the meantime the issue got solved recently through a support call.....

It looks like the upgrade process ran into a timeout while talking to Zookeeper. To proceed with the upgrade, the solution was to do "Pause Upgrade", go to Ambari Dashboard, check state of Zookeeper (which was fine, all up and running), then click "Resume Upgrade". After that the problem was gone and upgrade procedure continued.

Best regards, Gerd

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here