Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

performance tuning indexing storm topology

performance tuning indexing storm topology

Contributor

All,

Metron out of the box does not tune indexing topology and it's recommended to do so. I'm trying to follow this guideline

https://github.com/apache/metron/tree/master/metron-platform/metron-indexing

However; when I add additional partitions to the indexing topic, the newly added partition doesn't get consumed. Although, I did increased kafkaSpout in my indexing storm topology. How can I get storm to start another consumer and read off of the partition so that my offset is not in the unknown state any more? Any help is much appreciated.

GROUP  TOPIC  PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG   OWNER
indexing indexing 1 unknown 1074158 unknown consumer-1_/172.16.8.236
indexing indexing 0 82387393 2729661 -79657732 consumer-1_/172.16.8.237
Don't have an account?
Coming from Hortonworks? Activate your account here