Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

getkafka don't working in nifi

Solved Go to solution

getkafka don't working in nifi

Contributor

When we trying to use getkafka we see following error:

2016-10-07 17:37:39,469 INFO [pool-24-thread-1-EventThread] org.I0Itec.zkclient.ZkClient zookeeper state changed (Expired)

2016-10-07 17:37:39,470 INFO [ZkClient-EventThread-465-hdp-name1.lab.croc.ru:2181] k.consumer.ZookeeperConsumerConnector [95446e62-0157-1000-7951-fd4244e9aec2_###############-1475841346967-f0d261ce], exception during rebalance

kafka.common.KafkaException: Failed to parse the broker info from zookeeper: {"jmx_port":-1,"timestamp":"1475501559373","endpoints":["PLAINTEXT://############:6667"],"host":"#############","version":3,"port":6667}

next we see:

Caused by: kafka.common.KafkaException: Unknown version of broker registration. Only versions 1 and 2 are supported.{"jmx_port":-1,"timestamp":"1475501559373","endpoints":["PLAINTEXT://#########:6667"],"host":"##########","version":3,"port":6667}

Our hdp version is 2.5 and hdf version is 2.0.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: getkafka don't working in nifi

Master Guru

@Ramil Akhmadeev

HDP 2.5 comes with Apache Kafka 0.10.0.1. The NiFi getKafka processor uses the Kafka 0.8 client library. For communicating with Kafka 0.10 you should be using the consumeKafka_0_10 NiFi processor.

4 REPLIES 4

Re: getkafka don't working in nifi

Master Guru

@Ramil Akhmadeev

HDP 2.5 comes with Apache Kafka 0.10.0.1. The NiFi getKafka processor uses the Kafka 0.8 client library. For communicating with Kafka 0.10 you should be using the consumeKafka_0_10 NiFi processor.

Re: getkafka don't working in nifi

Contributor

Thank You for quick reply, can You tell me please where can I get consumeKafka_o_10 nifi processor?

Highlighted

Re: getkafka don't working in nifi

Contributor

I've found, thank You very much for solution!

Re: getkafka don't working in nifi

Master Guru

For other users/readers who do not know, HDF 2.0 includes as part of the release includes the following:

GetKafka and PutKafka --> Support Kafka 0.8

ConsumeKafka and PublishKafka --> Supports Kafka 0.9

ConsumeKafka_0_10 and PublishKafka_0_10 --> Supports Kafka 0.10

Thanks,

Matt

Don't have an account?
Coming from Hortonworks? Activate your account here