Reply
Explorer
Posts: 19
Registered: ‎04-23-2019

How to check the performance of a 5 node(256 gb) kafka cluster?

 
Highlighted
Cloudera Employee
Posts: 122
Registered: ‎03-01-2016

Re: How to check the performance of a 5 node(256 gb) kafka cluster?

Hi 

 

Thanks for posting the question to cloudera commuinity.

 

It would be a little too vargue to say how to check the performance. There are two things I'd suggest to do:

 

1) Benchmark the performance with the built-in load testing tools such as ConsumerPerformance and EndToEndLatecy.

 

2) Monitor the Kafka performance with the avaiable metrics:

https://www.cloudera.com/documentation/kafka/latest/topics/kafka_metrics.html

 

Additionally, you may want to check the cloudera documentations on how to tune kafka broker, consumer and producers for better performance:

 

https://www.cloudera.com/documentation/kafka/latest/topics/kafka_performance.html

 

Hope above information helps.

Announcements

Our community is getting a little larger. And a lot better.


Learn More about the Cloudera and Hortonworks community merger planned for late July and early August.