confluent sink connector consumer matrix through JMX

[#1]

I am using confluent HDFS sink connector and would like to know how to get consumer properties to expose through either JMS or rest api.

I checked the following two properties, however, I don't know how to expose matrix to jmx port

1. connect-standalone.properties

2. consumer.properties

[#2]

JMX is not exposed in a property file. It is toggled from two environment variable. Confluent and Apache Kafka installations share the same variables for this.

Source code here -

https://github.com/apache/kafka/blob/trunk/bin/kafka-run-class.sh#L166-L174

Basically, if you export the JMX_PORT to a valid port number, it will open that port for JMX monitoring of any Kafka-related script that you run.

The recommendation, however, is to use a cluster of machines running Connect Distributed, as it is a long-running process, and you can scale your metrics collection using tools like Prometheus JMX Exporter combined with a Grafana server for dashboarding.

[#3]

@Jordan Moore

This is what we have done. However, I am not able to see any consumer matrix , I am only able to see broker matrix. is there something I am doing wrong.

[#4]

@Gaurang Shah

If you see broker metrics, then that is where you exposed the port to. Kafka Connect is meant to be run on separate machines from the brokers, but if you are running on the same ones, then you must expose and monitor two differerent JMX ports.