Created on 02-24-2017 05:54 PM - edited 08-19-2019 03:11 AM
Following this steps for restarting Ambari Metrics I'm sticking on stopping Grafana.
And the background operation list
Should I kill it manually? Thanks in advance.
Created 02-24-2017 07:15 PM
I found the problem: the device that was filled up has this file
/var/lib/ambari-agent/data/structured-out-status.json
that differs with the other nodes. I following this steps as root
rm -f /var/lib/ambari-agent/data/structured-out-status.json ambari-agent restart
And I deleted the PID files in /var/run for applications that aren't responding for restart (like Zookeeper and Ambari Metrics Collector).
After that Ambari shows the process down. So I started them and now it works correctly.
Created 02-24-2017 06:05 PM
You can kill it from linux command from the server that run Ambari metrics and grafana. But also you may need to check the log why it is not able to shutdown.
pkill -KILL -u ams
Created 02-24-2017 06:24 PM
An application filled up the HDD and after the cleaning the log is corrupted (there are the last five lines)
2017/02/24 05:30:15 [I] Completed XXX.XXX.XXX.XXX - "GET / HTTP/1.1" 500 Internal Server Error 2528 bytes in 26900us 2017/02/24 05:31:15 [I] Completed XXX.XXX.XXX.XXX - "GET / HTTP/1.1" 500 Internal Server Error 2528 bytes in 14789us 2017/02/24 05:32:15 [I] Completed XXX.XXX.XXX.XXX - "GET / HTTP/1.1" 500 Internal Server Error 2528 bytes in 20252us 2017/02/24 05:33:15 [I] Completed XXX.XXX.XXX.XXX - "GET / HTTP/1.1" 500 Internal Server Error 2528 bytes in 16111us 2017/02
Created 02-24-2017 07:15 PM
I found the problem: the device that was filled up has this file
/var/lib/ambari-agent/data/structured-out-status.json
that differs with the other nodes. I following this steps as root
rm -f /var/lib/ambari-agent/data/structured-out-status.json ambari-agent restart
And I deleted the PID files in /var/run for applications that aren't responding for restart (like Zookeeper and Ambari Metrics Collector).
After that Ambari shows the process down. So I started them and now it works correctly.