GetKafka working for Kafka 0.10 but not ConsumeKafka or ConsumeKafka_0_10

[#1]

For some reason even though my Kafka version is listed as kafka_2.10-0.10.0.2.5.0.0-1245.jar (which I am pretty sure is Kafka 0.10), the only processor that works for me is GetKafka.

Consume_Kafka and Consume_Kafka_0_10 both will hang and not process any data... if I stop the nifi flow it does give an error seen in the screenshot. I have left it running up to 10 mins waiting for it to connect but it never seems to.

I have also including a screenshot of my KafkaConsume properties window which is using the same values as the ones I put in GetKafka when it works.

[#2 · Accepted solution]

Hi @Eric Lloyd,

Is your Kafka broker listening on port 2181? It looks like a zookeeper port.

As indicated in the properties description, ConsumeKafka processors are not using ZooKeeper to connect to the brokers as GetKafka is doing. If you provide a comma separated list of your brokers (with corresponding port), it should be OK.

Hope this helps.

[#3]

Crap. I thought the Zookeeper port and the Kafka brokers port were the same. Ill hunt down what ports my Kafka broker is listening on - not sure where to begin looking. Any suggestion? ( Im sure Ill find it through Google)

[#4]

With plaintext, I believe default port should be 9092.

[#5]

Got it! Thanks!

[#6]

Hi Eric, did you solved the issue?

I'm having the same problem when trying to use processors ConsumeKafka and ConsumeKafka_0_10 to read messages from kafka 0.10.1 (HDP 2.6 without kerberos). I did tests with different parameter settings and the processors starts without error but never receives the messages. I had same results executing the kafka-console-consumer shell with bootstrap-server parameter.

Can you help me? Any idea of what's its going on?