Skip to main content
CloudBoy
Silver ⭐️⭐️
February 21, 2019
Solved

In HDP3, Ambari backgound operations not able to restart stale config/restart of service in one of the master node

recently I have upgraded to HDP 3. If I run the stale config, it is not working(failing) to one of the master server(NN runnnig) all other nodes are working fine if I apply stale config.

Whatever the services running in this host are failing.

even i have restarted ambari server and ambari agent but no luck.

Please help


Thanks in advance..


jsensharma
Platinum ⭐️⭐️
February 21, 2019

@Giridharan C

1. On the problematic host are you able to start those service components manually? As described in the following doc?

https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.1.0/administration/content/starting_hdp_services.html

2. Have you restarted ambari-agent on that host and after restart do you see any Error in the

"/var/log/ambari-agent/ambari-agent.log"

3. Do you see the new command-xxx.json / output-xxx.txt and errors-xxx.txt kind of files are getting created in the Problematic host in the following location as soon as you trigger the stale config refresh command form ambari UI ?

Just check the newly created file as soon as you triger stal config refresh command from the UI.

# ls -lart /var/lib/ambari-agent/data/command*.json
# ls -lart /var/lib/ambari-agent/data/output-*.txt
# ls -lart /var/lib/ambari-agent/data/error*.txt

.

4. When the Stale Config update fails on the problematic node then in the Ambarti UI operational log do you see any error.

Can you please share that complete message which you see in the ambari UI when the stale config refresh was going on?

.

2 replies

jsensharma
Platinum ⭐️⭐️
Platinum ⭐️⭐️
February 21, 2019

@Giridharan C

1. On the problematic host are you able to start those service components manually? As described in the following doc?

https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.1.0/administration/content/starting_hdp_services.html

2. Have you restarted ambari-agent on that host and after restart do you see any Error in the

"/var/log/ambari-agent/ambari-agent.log"

3. Do you see the new command-xxx.json / output-xxx.txt and errors-xxx.txt kind of files are getting created in the Problematic host in the following location as soon as you trigger the stale config refresh command form ambari UI ?

Just check the newly created file as soon as you triger stal config refresh command from the UI.

# ls -lart /var/lib/ambari-agent/data/command*.json
# ls -lart /var/lib/ambari-agent/data/output-*.txt
# ls -lart /var/lib/ambari-agent/data/error*.txt

.

4. When the Stale Config update fails on the problematic node then in the Ambarti UI operational log do you see any error.

Can you please share that complete message which you see in the ambari UI when the stale config refresh was going on?

.

Translate
CloudBoy
CloudBoyAuthor
Silver ⭐️⭐️
February 26, 2019

after restarting ambari-agent it is working fine..

Translate