Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Additional Ambari metrics - Kafka

Solved Go to solution
Highlighted

Additional Ambari metrics - Kafka

Super Collaborator

I'm looking for a way to integrate some of the Kafka JMX metrics into Ambari monitoring and alerting if possible. The installation I will be working with is HDP 2.3.4, but at the moment I'm looking at the 2.4 sandbox. In the sandbox, I see these two files which are very similar:

/var/lib/ambari-server/resources/stacks/HDP/2.3/services/KAFKA/metrics.json

/var/lib/ambari-server/resources/common-services/KAFKA/0.8.1.2.2/metrics.json

They appear to be for collecting JMX metrics, but I'm not sure how to take advantage of this. Is it possible?

Thanks

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Additional Ambari metrics - Kafka

Expert Contributor

Ambari 2.2.2 comes with built in Grafana integration so you can build a dashboard of you choice. All metrics are self discovered and no extra steps are required.

Adding Grafana:

https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.2.0/bk_upgrading_Ambari/content/_adding_grafana...

Grafana docs:

http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.2.0/bk_Ambari_Users_Guide/content/_using_grafana...

The Ambari widgets provide a service level aggregated view of the service metrics. You would need to update the metrics.json file from "/var/lib/ambari-server/resources/stacks/HDP/2.3/services/KAFKA/metrics.json", this overrides the common-services definition for HDP versions 2.3+ And then add widget from Ambari.

Docs for that:

https://cwiki.apache.org/confluence/display/AMBARI/Enhanced+Service+Dashboard

View solution in original post

6 REPLIES 6
Highlighted

Re: Additional Ambari metrics - Kafka

Super Collaborator

These files have stuff like this in them, so how do I get this info in Ambari?

{
  "KAFKA_BROKER": {
    "Component": [
      {
        "type": "ganglia",
        "metrics": {
          "default": {
            "metrics/jvm/uptime": {
              "metric": "jvm.uptime",
              "pointInTime": true,
              "temporal": true
            },

/* -- SNIP -- */
            "metrics/kafka/server/BrokerTopicMetrics/AllTopicsBytesInPerSec/count": {
              "metric": "kafka.server.BrokerTopicMetrics.BytesInPerSec.count",
              "pointInTime": true,
              "temporal": true
            },
            "metrics/kafka/server/BrokerTopicMetrics/AllTopicsBytesOutPerSec/1MinuteRate": {
              "metric": "kafka.server.BrokerTopicMetrics.BytesOutPerSec.1MinuteRate",
              "pointInTime": true,
              "temporal": true
            },
            "metrics/kafka/server/BrokerTopicMetrics/AllTopicsBytesOutPerSec/5MinuteRate": {
              "metric": "kafka.server.BrokerTopicMetrics.BytesOutPerSec.5MinuteRate",
              "pointInTime": true,
              "temporal": true
            },
            "metrics/kafka/server/BrokerTopicMetrics/AllTopicsBytesOutPerSec/15MinuteRate": {
              "metric": "kafka.server.BrokerTopicMetrics.BytesOutPerSec.15MinuteRate",
              "pointInTime": true,
              "temporal": true
            },
            "metrics/kafka/server/BrokerTopicMetrics/AllTopicsBytesOutPerSec/meanRate": {
              "metric": "kafka.server.BrokerTopicMetrics.BytesOutPerSec.meanRate",
              "pointInTime": true,
              "temporal": true
            },
            "metrics/kafka/server/BrokerTopicMetrics/AllTopicsBytesOutPerSec/count": {
              "metric": "kafka.server.BrokerTopicMetrics.BytesOutPerSec.count",
              "pointInTime": true,
              "temporal": true
            },
            "metrics/kafka/controller/KafkaController/ActiveControllerCount": {
              "metric": "kafka.controller.KafkaController.ActiveControllerCount",
              "pointInTime": true,
              "temporal": true
            },

Highlighted

Re: Additional Ambari metrics - Kafka

Expert Contributor

Ambari 2.2.2 comes with built in Grafana integration so you can build a dashboard of you choice. All metrics are self discovered and no extra steps are required.

Adding Grafana:

https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.2.0/bk_upgrading_Ambari/content/_adding_grafana...

Grafana docs:

http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.2.0/bk_Ambari_Users_Guide/content/_using_grafana...

The Ambari widgets provide a service level aggregated view of the service metrics. You would need to update the metrics.json file from "/var/lib/ambari-server/resources/stacks/HDP/2.3/services/KAFKA/metrics.json", this overrides the common-services definition for HDP versions 2.3+ And then add widget from Ambari.

Docs for that:

https://cwiki.apache.org/confluence/display/AMBARI/Enhanced+Service+Dashboard

View solution in original post

Highlighted

Re: Additional Ambari metrics - Kafka

Super Collaborator

Thanks for the info @swagle.

We were able to convince the customer to upgrade to Ambari 2.2.2. They were thrilled with the fact that you can add widgets directly in the UI, and the fact that Grafana is available is icing on the cake. Incidentally, here is the JIRA for adding this for Kafka and Storm: https://issues.apache.org/jira/browse/AMBARI-14941

Highlighted

Re: Additional Ambari metrics - Kafka

Expert Contributor

KAFKA and STORM default dashboards are in the works and targeted for 2.4.0

However you can start visualizing these metrics with custom dashboard from 2.2.2 itself.

Highlighted

Re: Additional Ambari metrics - Kafka

Contributor

Hey @swagle..

Do you have any link for setting up Kafka custom dashboard in Grafana for Ambari 2.2.2?

Re: Additional Ambari metrics - Kafka

Expert Contributor

Actually, Grafana allow dashboard import. You might be able to just get the dashboard definition from Ambari source and import into you existing Grafana.

Here is link to trunk dashboard locations:

https://github.com/apache/ambari/tree/trunk/ambari-server/src/main/resources/common-services/AMBARI_...

Import is pretty straightforward:

http://docs.grafana.org/reference/export_import/

Don't have an account?
Coming from Hortonworks? Activate your account here