Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to use Apache Nifi ConsumeKafka_1_0 Processor

Highlighted

How to use Apache Nifi ConsumeKafka_1_0 Processor

New Contributor

I have Nifi cluster of and Kafka is also installed there.

Created one topic with 5 partitions, start consuming that topic with one gourpid. So that each partition will get unique messages.

Now I created the 5 ConsumeKafka_1_0 processors having the intent of getting unique messages on each consumer side. But only 2 of the ConsumeKafka_1_0 are consuming all the messages rest is setting ideal.

Now what I did is started the 5 command line Kafka consumer, and what happened is, I was able to see the all the partitions are getting the messages and able to consume them from command line consumer in round-robin fashion only.

Also, I tried descried the Kafka group and what I saw was only 2 of the Nifi ConsumeKafka_1_0 is consuming all the 5 partitions and rest is ideal, see the snapshot.

86457-capture.png

Would you please let me what I am doing wrong here with Nifi consumer processor.

Note - i used Nifi version is 1.5 and Kafka version is 1.0.

1 REPLY 1

Re: How to use Apache Nifi ConsumeKafka_1_0 Processor