Service Start/Stop Process Hangs on Ambari

[#1]

Hi Everyone, While I try to restart/start/stop any ambari service, it hangs everytime. I don't know the reason because no error, no warning in logs. Generally some steps hang at %9.

Here is the ss:

Do you have any idea why it hangs?

[#2]

@Berk Ardıç,

I have seen this error when there is not enough free memory in the node.

Can you try restarting the ambari agents in the nodes where you see this issue and try start/stop the service

ambari-server restart

Also, it will be good to check the output of 'free -m'

Thanks,

Aditya

[#3]

"not enough free memory" gave me an idea. In ambari metrics view screen, yarn memory seen %100. As I know it only allocate that memory not actually use it. Anyway I changed memory settings for yarn and reboot servers. Now I can do some changes and can restart any service. It seems ok for now.

Thank you so much

[#4]

Hi Again @Aditya Sirna,

Unfortunately, I realized that the problem is not about free memory for our environment. I changed yarn memory settings and restarted the servers. After 1 day the same problem occured again. The services are neither start nor stop. Is there any issue that reported like that?

Kind Regards

[#5]

@Berk Ardıç

Is this Hang being observed only during server start & stop or while performing any other operation on the mentioned host "vtmvixxxxxx" ?

.

If yes then please try restarting the ambari agent once on that host.

# ambari-agent stop
# ambari-agent start

The "Restart HBase Client" is hanging in the mentioned screen which can happen due to Repo unavailability as well on the mentioned host because the restart clients causes a quick access to the Hortonworks repo. So please check that there is not N/W issue while accessing the Hortonworks Repo from that host (or it may be your Local Offline HDP repo as well).

.

Still if you face this issue then please share the ambari-agent log as well as the "ambari-server.log" for quick review.

.

[#6 · Accepted solution]

If the standard process # service ambari-agent restart and # service ambari-server restart doesn't work try:

service ambari-agent stop
service ambari-server stop

sudo -u postgres psql
SELECT * FROM ambari.hostcomponentstate;
UPDATE ambari.hostcomponentstate SET current_state = 'INSTALLED' WHERE current_state = 'STOPPING';
UPDATE ambari.hostcomponentstate SET last_live_state = 'INSTALLED' WHERE last_live_state = 'STOPPING';
\q

service ambari-agent start
service ambari-server start

Start services in Ambari.

[#7]

thanks worked like a charm !!