Support Questions
Find answers, ask questions, and share your expertise

Cloudera kafka intergration with Promethous

Cloudera kafka intergration with Promethous

New Contributor



I am trying to integrate Prometheus with cloudera (5.16.0) kafka cluster (kafka version = 1.0.1), but unable to export matrices.


Steps performed:

Promethous Server:


tar -xzf prometheus-*.tar.gz
mv prometheus-2.3.2.linux-amd64 prometheus
rm prometheus-*.tar.gz
cd prometheus


nano prometheus.yml
scrape_interval: 10s
evaluation_interval: 10s
- job_name: 'kafka'
- targets:
- hadoop-2:8080 # Kafka 1 - change IP for your use case
- hadoop-3:8080 # Kafka 2 - change IP for your use case
- hadoop-4:8080 # Kafka 3 - change IP for your use case
- job_name: 'zookeeper'
- targets:
- hadoop-2:8080 #zookeeper
- hadoop-3:8080 #zookeeper
- hadoop-4:8080 #zookeeper



Kafka side:


cd /home/kumar
mkdir prometheus
cd prometheus

Downloaded jar --> jmx_prometheus_javaagent-0.3.1.jar

zookeeper-yml --> zookeeper.yaml

kafka-yml --> kafka-0-8-2.yml



Now, I wanted to add kafka & zookeeper environment as mention:

1. For Kafka



2. For Zookeeper:




So my question which (safety valve) will hold this environment configuration under Kafka & Zookeeper?


While I am configuring on plain Kafka cluster, I am adding all environment argument under /etc/systemd/system/kafka.service & /etc/systemd/system/zookeeper.service for Kafka & Zookeeper respectively.


Appreciate, If any one could help me on this as I am new to Cloudera.


Thank You.




Re: Cloudera kafka intergration with Promethous

Cloudera Employee

>> You can refer to the mentioned Git Repo[1] for information on Kafka exporter for Prometheus in Kafka Manager.




>> I would like to share information on SMM[2], Streams Messaging Manager is an operations monitoring and management tool from Cloudera that provides end-to-end visibility in an enterprise Apache Kafka environment.


With SMM, you can gain clear insights about your Kafka clusters. You can understand the end-to-end flow of message streams from producers to topics to consumers.