Ambari metrics display issue

[#1]

I have a HDP 2.3 cluster with 9 nodes: 1 management node, 2 master nodes, and 6 slave nodes.

The Ambari dashboard shows the metrics (cpu usage, disk usage, etc.) from the management and master nodes. The metrics from the slave nodes are not displayed (metrics service may not be installed or inaccessible). I have checked and the metrics monitor service is installed, running, and sending messages to the collector. Ntp is also running.

So why am I not seeing those metrics?

[#2]

@Rahul Tikekar

I am guessing that you are using FQDN .

Please see this.

http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.2.1/bk_ambari_reference_guide/content/_how_to_se...

[#3]

@Neeraj Sabharwal Yes we use FQDN and local repositories. So proxy server should not be an issue because the cluster has no Internet access. Also, all the nodes are set up identically so why do metrics from 3 hosts show and 6 don't? Thanks.

[#4]

@Rahul Tikekar

please check your SELinux and firewall settings.

[#5]

@Rahul Tikekar are you still having issues? Please accept the best answer or provide your own solution.

[#6]

I am not using a proxy server. The strange part is that the metrics from the edge node and two master servers are displayed but those from the data nodes are not.

[#7]

@Rahul Tikekar check firewall on those nodes and whether metrics monitors are up.

[#8]

The firewall on all the nodes is off. The metrics monitors are running on all nodes (I can even see the log file where the emitter.py is contacting the metrics collector).

[#9]

@Rahul Tikekar please review the ambari metrics user guide for troubleshooting tips. More troubleshooting on the wiki.

[#10 · Accepted solution]

@Rahul Tikekar

Try restarting your NTP servers !

syncing-multiple-servers.pdf

[#11]

Yes! Thank you @Geoffrey Shelton Okot. That did it - I followed the instructions in the document to sync the ntp servers and I'm in business. Thanks!

[#12]

thanks @Rahul Tikeker your answer was very helpful