getkafka don't working in nifi

[#1]

When we trying to use getkafka we see following error:

2016-10-07 17:37:39,469 INFO [pool-24-thread-1-EventThread] org.I0Itec.zkclient.ZkClient zookeeper state changed (Expired)

2016-10-07 17:37:39,470 INFO [ZkClient-EventThread-465-hdp-name1.lab.croc.ru:2181] k.consumer.ZookeeperConsumerConnector [95446e62-0157-1000-7951-fd4244e9aec2_###############-1475841346967-f0d261ce], exception during rebalance

kafka.common.KafkaException: Failed to parse the broker info from zookeeper: {"jmx_port":-1,"timestamp":"1475501559373","endpoints":["PLAINTEXT://############:6667"],"host":"#############","version":3,"port":6667}

next we see:

Caused by: kafka.common.KafkaException: Unknown version of broker registration. Only versions 1 and 2 are supported.{"jmx_port":-1,"timestamp":"1475501559373","endpoints":["PLAINTEXT://#########:6667"],"host":"##########","version":3,"port":6667}

Our hdp version is 2.5 and hdf version is 2.0.

[#2 · Accepted solution]

@Ramil Akhmadeev

HDP 2.5 comes with Apache Kafka 0.10.0.1. The NiFi getKafka processor uses the Kafka 0.8 client library. For communicating with Kafka 0.10 you should be using the consumeKafka_0_10 NiFi processor.

[#3]

Thank You for quick reply, can You tell me please where can I get consumeKafka_o_10 nifi processor?

[#4]

I've found, thank You very much for solution!

[#5]

For other users/readers who do not know, HDF 2.0 includes as part of the release includes the following:

GetKafka and PutKafka --> Support Kafka 0.8

ConsumeKafka and PublishKafka --> Supports Kafka 0.9

ConsumeKafka_0_10 and PublishKafka_0_10 --> Supports Kafka 0.10

Thanks,

Matt