question Re: Kafka Version on HDP-2.3.2? in Archives of Support Questions (Read Only)

Kafka Version on HDP-2.3.2?

muanang — Wed, 02 Mar 2016 10:55:26 GMT

Hi ALL!

We upgraded our cluster from HDP-2.2.0 to HDP-2.3.2

From HDP site, HDP-2.3.2 includes Kafka 0.8.2

But from our ambari-2.2 (hdp 2.3.2), kafka version show 0.9.0.2.

When I check kafka lib folder from broker kafka lib show 0.8.2 kafka version.

Which config setup should I follow, 0.9.x or 0.8.2?

Thank you!

Re: Kafka Version on HDP-2.3.2?

aervits — Wed, 02 Mar 2016 10:57:05 GMT

0.9.0.2 sounds like a bug, I'd follow 0.8.2 @Michael Dennis Uanang

Re: Kafka Version on HDP-2.3.2?

muanang — Wed, 02 Mar 2016 11:07:00 GMT

@Artem Ervits thanks for the quick reply.

Done a quick checking of kafka config on ambari.

And now I'm checking, 0.8.2 documentation.

On ambari, theres a parameter call "listeners=PLAINTEXT:///localhost:6667" on our kafka settings.

But when I check on 0.8.2 documentation/broke-config, there's no "listeners" option available, but available in 0.9.x documentation though.

Re: Kafka Version on HDP-2.3.2?

nsabharwal — Wed, 02 Mar 2016 12:32:00 GMT

@Michael Dennis Uanang

See this http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_upgrading_hdp_manually/content/upgrade_kafka.html

Re: Kafka Version on HDP-2.3.2?

nsabharwal — Wed, 02 Mar 2016 13:18:38 GMT

   hdp-select set kafka-broker version

Re: Kafka Version on HDP-2.3.2?

muanang — Thu, 03 Mar 2016 09:40:39 GMT

@Neeraj Sabharwal

Actually we did the Rolling Upgrade for HDR-2.3.2.

Re: Kafka Version on HDP-2.3.2?

nsabharwal — Sat, 05 Mar 2016 13:35:43 GMT

@Michael Dennis Uanang I did notice that and that's why you may have to run that statement

Re: Kafka Version on HDP-2.3.2?

muanang — Mon, 07 Mar 2016 09:59:18 GMT

Thanks @Neeraj Sabharwal! You the man!

Do I still need to run the command manually? Sorry just need to clear it out. 🙂

Re: Kafka Version on HDP-2.3.2?

nsabharwal — Mon, 07 Mar 2016 10:02:13 GMT

@Michael Dennis Uanang If version is not up to date the Yes 🙂

Re: Kafka Version on HDP-2.3.2?

muanang — Mon, 07 Mar 2016 10:55:05 GMT

How can I verify if kafka is successfully updated after HDP rolling upgrade?

Re: Kafka Version on HDP-2.3.2?

nsabharwal — Mon, 07 Mar 2016 10:59:29 GMT

@Michael Dennis Uanang Try this

[root@sandbox ~]# rpm -qa | grep kafka

kafka_2_4_0_0_169-0.9.0.2.4.0.0-169.el6.noarch

ranger_2_4_0_0_169-kafka-plugin-0.5.0.2.4.0.0-169.el6.x86_64

[root@sandbox ~]#

[root@sandbox bin]# hdp-select | grep kafka

kafka-broker - 2.4.0.0-169

[root@sandbox bin]#

Also, run a validation test

"To verify that the Kafka daemon joined the cluster, create a topic and submit it to Kafka. Send a test message for that topic, and then validate that it was received by a consumer."

Re: Kafka Version on HDP-2.3.2?

muanang — Mon, 07 Mar 2016 11:13:18 GMT

All brokers are all updated.

r6r41kafka.xxx | >> kafka-broker - 2.3.2.0-2950

r7r41kafka.xxx | >> kafka-broker - 2.3.2.0-2950

r8r41kafka.xxx | >> kafka-broker - 2.3.2.0-2950

Thanks again @Neeraj Sabharwal!

Re: Kafka Version on HDP-2.3.2?

muanang — Wed, 16 Mar 2016 09:31:49 GMT

So is this kafka 0.8.2 or 0.9.0? Just want to clarify. 🙂

Re: Kafka Version on HDP-2.3.2?

muanang — Wed, 23 Mar 2016 07:29:21 GMT

Kafka version on HDP 2.3.2 is v0.8.2 as confirmed by Subhasis, Ratish and Dev from Hortonworks.