There is a datacenter wide Ganglia monitoring wherein all the clusters are HDP 2.2. Now we are considering HDP 2.4 for a couple of new clusters. How do we integrate ambari metrics with datacenter wide ganglia

[#1]

There is a DC wide Ganglia monitoring in place where in the clusters have been HDP 2.2 as of today. Now some of the new clusters need to have HDP 2.4. HDP 2.4 does not support Ganglia. Is there a way by which Ambari metrics can push data to Ganglia for the details needed by Ganglia.

[#2 · Accepted solution]

Even though we no longer support Ganglia in our stack, the ganglia metrics sink in Hadoop is not removed, you can still leverage that https://wiki.apache.org/hadoop/GangliaMetrics

Another option is DropWizard https://community.hortonworks.com/repos/56568/dropwizard-metrics-reporter-for-apache-hadoop-metr.htm...

AMS supports rest API so you can also build your own sink, I tried similar approach with Apache Nifi And Graphite.

[#3]

Thanks Artem Ervits. We were considering to extend the HadoopSink class for our purpose - say GangliaSink. The instance of this GangliaSink class will run in each of the machines (similar to the gmond of Ganglia). In this case, we may have to probably install gmetad in one of the machines in the cluster. We will check out the options provided by you too. Thank you so much.

[#4]

@Mohana Murali Gurunathan it would certainly be of benefit for the wider community when you implement your solution to submit an article on HCC on this. I'm sure there are others that would be of interest to see how it's done. Thank you

[#5]

@Artem Ervits will definitely submit an article once it is implemented.