question Re: There is a datacenter wide Ganglia monitoring wherein all the clusters are HDP 2.2. Now we are considering HDP 2.4 for a couple of new clusters. How do we integrate ambari metrics with datacenter wide ganglia in Support Questions

There is a datacenter wide Ganglia monitoring wherein all the clusters are HDP 2.2. Now we are considering HDP 2.4 for a couple of new clusters. How do we integrate ambari metrics with datacenter wide ganglia

mohanamurali_gu — Wed, 02 Nov 2016 23:32:54 GMT

There is a DC wide Ganglia monitoring in place where in the clusters have been HDP 2.2 as of today. Now some of the new clusters need to have HDP 2.4. HDP 2.4 does not support Ganglia. Is there a way by which Ambari metrics can push data to Ganglia for the details needed by Ganglia.

Re: There is a datacenter wide Ganglia monitoring wherein all the clusters are HDP 2.2. Now we are considering HDP 2.4 for a couple of new clusters. How do we integrate ambari metrics with datacenter wide ganglia

aervits — Sat, 05 Nov 2016 09:25:12 GMT

Even though we no longer support Ganglia in our stack, the ganglia metrics sink in Hadoop is not removed, you can still leverage that https://wiki.apache.org/hadoop/GangliaMetrics

Another option is DropWizard https://community.hortonworks.com/repos/56568/dropwizard-metrics-reporter-for-apache-hadoop-metr.html

AMS supports rest API so you can also build your own sink, I tried similar approach with Apache Nifi And Graphite.

Re: There is a datacenter wide Ganglia monitoring wherein all the clusters are HDP 2.2. Now we are considering HDP 2.4 for a couple of new clusters. How do we integrate ambari metrics with datacenter wide ganglia

mohanamurali_gu — Tue, 08 Nov 2016 13:54:14 GMT

Thanks Artem Ervits. We were considering to extend the HadoopSink class for our purpose - say GangliaSink. The instance of this GangliaSink class will run in each of the machines (similar to the gmond of Ganglia). In this case, we may have to probably install gmetad in one of the machines in the cluster. We will check out the options provided by you too. Thank you so much.

Re: There is a datacenter wide Ganglia monitoring wherein all the clusters are HDP 2.2. Now we are considering HDP 2.4 for a couple of new clusters. How do we integrate ambari metrics with datacenter wide ganglia

aervits — Tue, 08 Nov 2016 23:58:58 GMT

@Mohana Murali Gurunathan it would certainly be of benefit for the wider community when you implement your solution to submit an article on HCC on this. I'm sure there are others that would be of interest to see how it's done. Thank you

Re: There is a datacenter wide Ganglia monitoring wherein all the clusters are HDP 2.2. Now we are considering HDP 2.4 for a couple of new clusters. How do we integrate ambari metrics with datacenter wide ganglia

mohanamurali_gu — Wed, 09 Nov 2016 12:35:05 GMT

@Artem Ervits will definitely submit an article once it is implemented.