Consume Kafka processor not working

[#1]

consumer-processors.png

Hi,
I have HDF installed on my server. I want consume data from a kafka topic and write it to a file which can then be loaded into HDFS.

The problem is that the ConsumeKafka processor doesnt consume an messages from the topic, I tried different processors and also verified the hostname/ports. I added the group id by listing it from the zookeeper shell i.e. ls /consumers in the zookeeper shell and added shown group id in the processors group id.

THings I have done:

Verified the hostname and ports

Verified that data is consumed using the kafka-console-consumer.sh

Googled a lot!

Any help will be greatly appreciated.

I have attached the pics for reference

Thanks

kafka-consuemr-properties.png

[#2]

@Hamza Rashid

Change the below property value to

Offset Reset

earliest //automatically reset the offset to earliest

And if you are already consumed from the Group then change the

Group ID

nifi-cosumers-1

Run the processor again.

-

If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of issues.

[#3]

No effect whatsoever.

[#4]

Nope, it still doesn't work.

[#5]

@Hamza Rashid This should be broker hostname and port(default 6667, "listeners" property will have the configured port in your environment) instead of zookeeper port.

[#6]

@Hamza Rashid Did the above mentioned changes helped in addressing your issue?

[#7]

No sir, no luck yet.

[#8]

Can you please recommend some resources that might help me in this?

[#9 · Accepted solution]

hy @Hamza,
if you are using no security protocol like kerberos/tsl then do use kafka processor version less than 0.9 which is getkafka and for those having kerberos security installed must have to use version above than 0.9 like consumekafka 0.10 and above.Its mandatroy.

[#10]

Thank you.