Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

moved metrics collector, missing metrics

Highlighted

Re: moved metrics collector, missing metrics

New Contributor

Hmm, it works now. Have no idea why. I increased heapsize for AMS Collector, restarted required services and it seems to work. But I didn't check if the grahps was working before doing so. However I just did this, and in Grafana I have over 5 hours of data. So I don't know if it's the heapsize that I adjusted just now or that it fixed itself earlier.

Suspiciously I restarted a datanode+NFS client 5 hours ago, roughly around the same time from when I have historic data from.

Re: moved metrics collector, missing metrics

@Elias Abacioglu

The screenshot indicates that dashboard show the last hour metrics. During the migration, if Ambari metrics were stopped, there was no data collection. Is it possible that you actually did not have available that during that gap timeframe. Is this still a problem today?

Re: moved metrics collector, missing metrics

New Contributor

I am currently running Ambari 2.4.0 and was experiencing this exact same issue. I verified all my settings were correct and pointing to the new AMS collector. Unfortunately I had to restart/failover the HDFS NameNodes and YARN Resource Manager to resolve this issue.

Re: moved metrics collector, missing metrics

Expert Contributor

This is unfortunately true because of sinks that run in the Hadoop daemon's jvm and can only read the new collector from configs upon restart. In the latest versions of Ambari (2.5+) with AMS HA (https://issues.apache.org/jira/browse/AMBARI-15901), the sinks fallback on zookeeper to discover collector shard making the move collector operation seemless.

Don't have an account?
Coming from Hortonworks? Activate your account here