question Re: GetKafka working for Kafka 0.10 but not ConsumeKafka or ConsumeKafka_0_10 in Support Questions

GetKafka working for Kafka 0.10 but not ConsumeKafka or ConsumeKafka_0_10

elloyd — Mon, 19 Aug 2019 07:59:27 GMT

For some reason even though my Kafka version is listed as kafka_2.10-0.10.0.2.5.0.0-1245.jar (which I am pretty sure is Kafka 0.10), the only processor that works for me is GetKafka.

Consume_Kafka and Consume_Kafka_0_10 both will hang and not process any data... if I stop the nifi flow it does give an error seen in the screenshot. I have left it running up to 10 mins waiting for it to connect but it never seems to.

I have also including a screenshot of my KafkaConsume properties window which is using the same values as the ones I put in GetKafka when it works.

Re: GetKafka working for Kafka 0.10 but not ConsumeKafka or ConsumeKafka_0_10

pvillard — Wed, 08 Mar 2017 03:03:15 GMT

Hi @Eric Lloyd,

Is your Kafka broker listening on port 2181? It looks like a zookeeper port.

As indicated in the properties description, ConsumeKafka processors are not using ZooKeeper to connect to the brokers as GetKafka is doing. If you provide a comma separated list of your brokers (with corresponding port), it should be OK.

Hope this helps.

Re: GetKafka working for Kafka 0.10 but not ConsumeKafka or ConsumeKafka_0_10

elloyd — Wed, 08 Mar 2017 03:06:46 GMT

Crap. I thought the Zookeeper port and the Kafka brokers port were the same. Ill hunt down what ports my Kafka broker is listening on - not sure where to begin looking. Any suggestion? ( Im sure Ill find it through Google)

Re: GetKafka working for Kafka 0.10 but not ConsumeKafka or ConsumeKafka_0_10

pvillard — Wed, 08 Mar 2017 03:08:05 GMT

With plaintext, I believe default port should be 9092.

Re: GetKafka working for Kafka 0.10 but not ConsumeKafka or ConsumeKafka_0_10

elloyd — Wed, 08 Mar 2017 03:09:57 GMT

Got it! Thanks!

Re: GetKafka working for Kafka 0.10 but not ConsumeKafka or ConsumeKafka_0_10

nahuel-tarello — Thu, 07 Dec 2017 23:47:27 GMT

Hi Eric, did you solved the issue?

I'm having the same problem when trying to use processors ConsumeKafka and ConsumeKafka_0_10 to read messages from kafka 0.10.1 (HDP 2.6 without kerberos). I did tests with different parameter settings and the processors starts without error but never receives the messages. I had same results executing the kafka-console-consumer shell with bootstrap-server parameter.

Can you help me? Any idea of what's its going on?