Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

getkafka don't working in nifi

Contributor

When we trying to use getkafka we see following error:

2016-10-07 17:37:39,469 INFO [pool-24-thread-1-EventThread] org.I0Itec.zkclient.ZkClient zookeeper state changed (Expired)

2016-10-07 17:37:39,470 INFO [ZkClient-EventThread-465-hdp-name1.lab.croc.ru:2181] k.consumer.ZookeeperConsumerConnector [95446e62-0157-1000-7951-fd4244e9aec2_###############-1475841346967-f0d261ce], exception during rebalance

kafka.common.KafkaException: Failed to parse the broker info from zookeeper: {"jmx_port":-1,"timestamp":"1475501559373","endpoints":["PLAINTEXT://############:6667"],"host":"#############","version":3,"port":6667}

next we see:

Caused by: kafka.common.KafkaException: Unknown version of broker registration. Only versions 1 and 2 are supported.{"jmx_port":-1,"timestamp":"1475501559373","endpoints":["PLAINTEXT://#########:6667"],"host":"##########","version":3,"port":6667}

Our hdp version is 2.5 and hdf version is 2.0.

1 ACCEPTED SOLUTION

Master Guru

@Ramil Akhmadeev

HDP 2.5 comes with Apache Kafka 0.10.0.1. The NiFi getKafka processor uses the Kafka 0.8 client library. For communicating with Kafka 0.10 you should be using the consumeKafka_0_10 NiFi processor.

View solution in original post

4 REPLIES 4

Master Guru

@Ramil Akhmadeev

HDP 2.5 comes with Apache Kafka 0.10.0.1. The NiFi getKafka processor uses the Kafka 0.8 client library. For communicating with Kafka 0.10 you should be using the consumeKafka_0_10 NiFi processor.

Contributor

Thank You for quick reply, can You tell me please where can I get consumeKafka_o_10 nifi processor?

Contributor

I've found, thank You very much for solution!

Master Guru

For other users/readers who do not know, HDF 2.0 includes as part of the release includes the following:

GetKafka and PutKafka --> Support Kafka 0.8

ConsumeKafka and PublishKafka --> Supports Kafka 0.9

ConsumeKafka_0_10 and PublishKafka_0_10 --> Supports Kafka 0.10

Thanks,

Matt