Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

getkafka don't working in nifi

avatar
Rising Star

When we trying to use getkafka we see following error:

2016-10-07 17:37:39,469 INFO [pool-24-thread-1-EventThread] org.I0Itec.zkclient.ZkClient zookeeper state changed (Expired)

2016-10-07 17:37:39,470 INFO [ZkClient-EventThread-465-hdp-name1.lab.croc.ru:2181] k.consumer.ZookeeperConsumerConnector [95446e62-0157-1000-7951-fd4244e9aec2_###############-1475841346967-f0d261ce], exception during rebalance

kafka.common.KafkaException: Failed to parse the broker info from zookeeper: {"jmx_port":-1,"timestamp":"1475501559373","endpoints":["PLAINTEXT://############:6667"],"host":"#############","version":3,"port":6667}

next we see:

Caused by: kafka.common.KafkaException: Unknown version of broker registration. Only versions 1 and 2 are supported.{"jmx_port":-1,"timestamp":"1475501559373","endpoints":["PLAINTEXT://#########:6667"],"host":"##########","version":3,"port":6667}

Our hdp version is 2.5 and hdf version is 2.0.

1 ACCEPTED SOLUTION

avatar
Master Mentor

@Ramil Akhmadeev

HDP 2.5 comes with Apache Kafka 0.10.0.1. The NiFi getKafka processor uses the Kafka 0.8 client library. For communicating with Kafka 0.10 you should be using the consumeKafka_0_10 NiFi processor.

View solution in original post

4 REPLIES 4

avatar
Master Mentor

@Ramil Akhmadeev

HDP 2.5 comes with Apache Kafka 0.10.0.1. The NiFi getKafka processor uses the Kafka 0.8 client library. For communicating with Kafka 0.10 you should be using the consumeKafka_0_10 NiFi processor.

avatar
Rising Star

Thank You for quick reply, can You tell me please where can I get consumeKafka_o_10 nifi processor?

avatar
Rising Star

I've found, thank You very much for solution!

avatar
Master Mentor

For other users/readers who do not know, HDF 2.0 includes as part of the release includes the following:

GetKafka and PutKafka --> Support Kafka 0.8

ConsumeKafka and PublishKafka --> Supports Kafka 0.9

ConsumeKafka_0_10 and PublishKafka_0_10 --> Supports Kafka 0.10

Thanks,

Matt