Support Questions
Find answers, ask questions, and share your expertise

Is there any way to create own kafka producer,consumer and kafka brokers performance metrics and pass the logs into Elasticsearch

Rising Star

I am aware of Ambari has all hadoop system performance metrices and dashboards in Grafana . But I looking to construct my own kafka producers,kafka consumers and kafka brokers metrics in Kibana so i have to log the system performance logs into Elasticsearch.So how can we approach this idea to start it? I know something like Mbean which has the system stats values ..Using Mbeans can we create metrics of the kafka brokers,producers and consumers

1 REPLY 1

Guru

Hello @Adhishankar Nanjundan ,

just an architectural idea for your approach.

What about putting all your metrics in one dedicated topic itself, then use Kafka-Connect to insert data from that topic into ElasticSearch....by using Kafka-Connect ElasticSearch sink ?!?!

Regards, Gerd

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.