Reply
Explorer
Posts: 19
Registered: ‎04-23-2019

How to check the performance of a 5 node(256 gb) kafka cluster?

 
Cloudera Employee
Posts: 119
Registered: ‎03-01-2016

Re: How to check the performance of a 5 node(256 gb) kafka cluster?

Hi 

 

Thanks for posting the question to cloudera commuinity.

 

It would be a little too vargue to say how to check the performance. There are two things I'd suggest to do:

 

1) Benchmark the performance with the built-in load testing tools such as ConsumerPerformance and EndToEndLatecy.

 

2) Monitor the Kafka performance with the avaiable metrics:

https://www.cloudera.com/documentation/kafka/latest/topics/kafka_metrics.html

 

Additionally, you may want to check the cloudera documentations on how to tune kafka broker, consumer and producers for better performance:

 

https://www.cloudera.com/documentation/kafka/latest/topics/kafka_performance.html

 

Hope above information helps.

Announcements
New solutions