Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to check the performance of a 5 node(256 gb) kafka cluster?

How to check the performance of a 5 node(256 gb) kafka cluster?

Explorer
 
1 REPLY 1

Re: How to check the performance of a 5 node(256 gb) kafka cluster?

Contributor

Hi 

 

Thanks for posting the question to cloudera commuinity.

 

It would be a little too vargue to say how to check the performance. There are two things I'd suggest to do:

 

1) Benchmark the performance with the built-in load testing tools such as ConsumerPerformance and EndToEndLatecy.

 

2) Monitor the Kafka performance with the avaiable metrics:

https://www.cloudera.com/documentation/kafka/latest/topics/kafka_metrics.html

 

Additionally, you may want to check the cloudera documentations on how to tune kafka broker, consumer and producers for better performance:

 

https://www.cloudera.com/documentation/kafka/latest/topics/kafka_performance.html

 

Hope above information helps.