question Re: Ambari shows the Zookeeper quorum as down but Zookeeper is running on all quorum nodes. Can I force Ambari to refresh? in Archives of Support Questions (Read Only)

Ambari shows the Zookeeper quorum as down but Zookeeper is running on all quorum nodes. Can I force Ambari to refresh?

TerryP — Wed, 18 Nov 2015 04:57:09 GMT

I want to avoid using the REST API to force the status. And even though the service is marked as down, there is no option to run a service check.

Re: Ambari shows the Zookeeper quorum as down but Zookeeper is running on all quorum nodes. Can I force Ambari to refresh?

jstraub — Wed, 18 Nov 2015 05:01:51 GMT

You could try to restart the Ambari Agents on the Zookeeper nodes or the complete ambari server. What is the current state of the Zookeeper service and host_components? could you check via Rest API <ambari-server>/api/v1/clusters/<clustername>/services/ZOOKEEPER

Re: Ambari shows the Zookeeper quorum as down but Zookeeper is running on all quorum nodes. Can I force Ambari to refresh?

nsabharwal — Wed, 18 Nov 2015 05:17:23 GMT

@terry

clear the browser cache and check the status. Please post the log entries from ambari-server.log related to zk.

Re: Ambari shows the Zookeeper quorum as down but Zookeeper is running on all quorum nodes. Can I force Ambari to refresh?

TerryP — Wed, 18 Nov 2015 07:00:30 GMT

I pulled the status and everything showed up as started! The Ambari agents and server were then cycled with no change in status.

Re: Ambari shows the Zookeeper quorum as down but Zookeeper is running on all quorum nodes. Can I force Ambari to refresh?

TerryP — Wed, 18 Nov 2015 07:03:06 GMT

The incorrect status was caused by the contents of the zookeeper_server.pid file being out of sync with the running zookeeper process. Updating the file with the proper PID allowed the correct Zookeeper status to be displayed in Ambari.

Re: Ambari shows the Zookeeper quorum as down but Zookeeper is running on all quorum nodes. Can I force Ambari to refresh?

nsabharwal — Wed, 18 Nov 2015 08:36:41 GMT

@terry Did you update the pid manually?

Re: Ambari shows the Zookeeper quorum as down but Zookeeper is running on all quorum nodes. Can I force Ambari to refresh?

bsaini — Wed, 18 Nov 2015 21:51:25 GMT

Sounds like this could be a bug..

Re: Ambari shows the Zookeeper quorum as down but Zookeeper is running on all quorum nodes. Can I force Ambari to refresh?

jstraub — Wed, 18 Nov 2015 22:21:10 GMT

The PID-file problem is actually quite common. I have seen this a couple of times already, not just zookeeper-related (e.g. services not starting because of existing pid file with invalid process id, ...).

Re: Ambari shows the Zookeeper quorum as down but Zookeeper is running on all quorum nodes. Can I force Ambari to refresh?

orenault — Wed, 18 Nov 2015 22:41:49 GMT

@Jonas Straub when you see this behaviour could you make sure to log a ticket with our support team.

FYI, I've asked support to get the ZK issue in a ticket.

Re: Ambari shows the Zookeeper quorum as down but Zookeeper is running on all quorum nodes. Can I force Ambari to refresh?

jstraub — Wed, 18 Nov 2015 22:45:28 GMT

@Olivier Renault I usually do 🙂

Re: Ambari shows the Zookeeper quorum as down but Zookeeper is running on all quorum nodes. Can I force Ambari to refresh?

bsaini — Wed, 18 Nov 2015 22:46:24 GMT

@Jonas Straub I've run into that problem also but the issue I was referring to was the fact that Ambari was reporting incorrect status of the process. I think in situations when the PID file already exists the error message is very direct and indicates exactly what the problem is (which is good)

Re: Ambari shows the Zookeeper quorum as down but Zookeeper is running on all quorum nodes. Can I force Ambari to refresh?

jstraub — Wed, 18 Nov 2015 22:53:13 GMT

You're right that might be a bug and should be raised with the engineering/support team.