Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDP 2.6 ( kafka 0.10.1.2.6.3.0)And Confluent Control Center ( confluent 3.3.0)

Highlighted

HDP 2.6 ( kafka 0.10.1.2.6.3.0)And Confluent Control Center ( confluent 3.3.0)

Explorer

Trying to monitor My hdp 2.6 kafka cluster with Ambari, ( also looking to the grafana dashboards)

i do not find any feature allowing to monitor the cluster at a fine grained level

( i mean managing end2end delivery , reacting with alerts, or perfs management --> all from a GUI

as the Control Center (from Confluence) allows.

so i tried iinstall/set up the CCC on my HDP and i experiment some start up issues despite the changes in properties ( kafka via ambari) and control center properties ,
i get the rocket ship graphic in the web interface ( see ) https://docs.confluent.io/current/control-center/docs/installation/troubleshooting.html#system-healt...

Before going further , i was wondering about compatiblity issues bettween CCC and HDP kafka

My HW/SW context

Confluent Control Center ( confluent 3.3.0) + HDP 2.6 ( kafka 0.10.1.2.6.3.0) on a linux VM ubuntu 16.04.2
BR

Philg

3 REPLIES 3
Highlighted

Re: HDP 2.6 ( kafka 0.10.1.2.6.3.0)And Confluent Control Center ( confluent 3.3.0)

Super Collaborator

Confluent Control Center requires an enterprise license in the long-term, which if you are using, you probably should install and maintain your own Kafka installation outside of HDP; maybe sharing a Zookeeper.

In any case, Confluent 3.3.0 is for Kafka version 0.11.0.0, which means your brokers are old. Also, the Kafka topic that control center listens to requires having Confluent's packages on your Kafka classpath. Therefore, while maybe possible, it needs to be distributed on all Kafka brokers and Connect workers, but Ambari isn't going to do that just by changing some configurations.

If you want some type of alerting and performance monitoring, simply exposing JMX metrics can provide lots of useful information like In-Sync Replicas, Partition Count per broker, Total Bytes In/Out, message throughput. For example, see https://www.robustperception.io/monitoring-kafka-with-prometheus/

Re: HDP 2.6 ( kafka 0.10.1.2.6.3.0)And Confluent Control Center ( confluent 3.3.0)

Explorer

Thanks Jordan

i was not aware about the constraint regarding 0.11 kafka version.

you are right, an approach with JMX metrics exploited through external Apps can be nice for

the long term ..

Highlighted

Re: HDP 2.6 ( kafka 0.10.1.2.6.3.0)And Confluent Control Center ( confluent 3.3.0)

Super Collaborator
@phil gib

If you want to try using Control Center, you can use version 3.1.2, which is for brokers running 0.10.1.1

https://docs.confluent.io/3.1.2/control-center/docs/index.html

Don't have an account?
Coming from Hortonworks? Activate your account here