Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Deleted Hosts still appear in Grafana

Solved Go to solution

Deleted Hosts still appear in Grafana

Expert Contributor

Hello all

I've noticed that in the Grafana UI when i filter for specific servers i can still see in the list Hosts that are no longer a part of the cluster (Datanodes that were deleted completely from the cluster).
How can i remove them from Grafana UI ?

Adi

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Deleted Hosts still appear in Grafana

Super Mentor

@Adi Jabkowsky

Grafana collects information from Ambari Metrics collector.

Ambari metrics collecotr stores hisgtorical metrics data. So your Old Host data will still be there.

If you want to verify your host removal changes then click at the top right corner of the Grafana UI "Last 6 Hours" data to change the time after you deleted the Host.

Other options may be to set the TTL value for the host level metrics (timeline.metrics.host.aggregator.daily.ttl) to a small value so that older data from the specified date is not maintained inside the AMS database (However this approach will cause deletion of other host metrics from the specified date)

Please see this link for more information on TTL: https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.0.0/bk_ambari-operations/content/ams_aggregated...

4 REPLIES 4

Re: Deleted Hosts still appear in Grafana

Super Mentor

@Adi Jabkowsky

Grafana collects information from Ambari Metrics collector.

Ambari metrics collecotr stores hisgtorical metrics data. So your Old Host data will still be there.

If you want to verify your host removal changes then click at the top right corner of the Grafana UI "Last 6 Hours" data to change the time after you deleted the Host.

Other options may be to set the TTL value for the host level metrics (timeline.metrics.host.aggregator.daily.ttl) to a small value so that older data from the specified date is not maintained inside the AMS database (However this approach will cause deletion of other host metrics from the specified date)

Please see this link for more information on TTL: https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.0.0/bk_ambari-operations/content/ams_aggregated...

Re: Deleted Hosts still appear in Grafana

Expert Contributor

@Jay Kumar SenSharma

Thank you for your promptly response !
I was refering to hosts that were deleted months ago that are still there...
So it seems that Grafana isn't aware to the change....

Re: Deleted Hosts still appear in Grafana

Super Mentor

@Adi Jabkowsky

Grafana pulls all historical metrics related data from AMS collector, So your AMS might be having the old host information.

Please try accessing the following URL of AMS collector to see how many hosts are there and to verify the AMS database still has Old host info:

http://<ams-host>:6188/ws/v1/timeline/metrics/hosts

Re: Deleted Hosts still appear in Grafana

Expert Contributor

@Jay Kumar SenSharma

The old hosts are still in the AMS metrics. I will take care of the TTL in AMS as per your recommendations.
Thank you !

Don't have an account?
Coming from Hortonworks? Activate your account here