Garafana faild to start

[#1]

Dears,

I need your help in fixing Grafana relevant issue: when I start services using Ambari, all start successfully except Grafana. And after it fails to start, all other services in the same host fail also. While I was troubleshooting, I found that :

1) Process id in "grafana-server.pid" is not matching the actual Id which is shown from "ps aux | grep grafana". After changing it manaully in grafana-server.pid, it changes back to mismatched PID after I restart Grafana.

2) in "/var/log/ambari-metrics-grafana/log", it says that "web.go:93 StartServer()] [E] Fail to start server: listen tcp 0.0.0.0:3000: bind: address already in use"

Please your kind help?

grafana-error-1377.txt

grafana-error-output-1377.txt

grafana-log.txt

[#2 · Accepted solution]

Dears,

Just to update:

I found the solution in https://community.hortonworks.com/articles/18088/ambari-shows-hdp-services-to-be-down-whereas-they.h...

"One of the issue could be due to /var/lib/ambari-agent/data/structured-out-status.json. Cat this file to review the content. Typical content could be like following:

cat structured-out-status.json {"processes": [], "securityState": "UNKNOWN"}

Compare the content with the same file in another node which is working fine.

Stop ambari-agent, move this file to another file and restart ambari-agent."

After that I restarted ambari-agent on the relevant host and things working back properly.

[#3]

While grafana service is stopped on Ambari. I find grafana process with.

ps aux | grep grafana

Then kill the process.

Started Grafana from Ambari.