Kafka 1.0.0.3.1 on HDP 2.6.4.0-91 & HDF 3.1.1.0

[#1]

We want to install Kafka 1.0.0.3.1 on our HDP 2.6.4.0-91 cluster. We have installed HDF 3.1.1.0 management pack on ambari server, but still it only shows NiFI and NiFI Registry, Kafka version shown under "Add services" is older i.e. 0.10

Any suggestion how to add Kafka 1.0.0.3.1 on HDP 2.6.4.0-91 ? @Jay Kumar SenSharma

[#2]

Hi @Ankur Gupta

Using the same Ambari for HDF and HDP have some caveat with the latest version.

You cannot install SAM and Schema Registry for HDF 3.1 on an HDP 2.6.4 cluster, and you cannot upgrade your Storm and Kafka versions if they exist on an HDP cluster. This is a temporary limitation with HDF 3.1 and will be resolved with HDP 3.0 and HDF 3.2. This has been described in the doc

https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.1.1/bk_planning-your-deployment/content/ch_depl...

https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.1.1/bk_installing-hdf-on-hdp/content/ch_add-hdf...

To have Kafka 1.x in HDP you need to wait for HDP 3.0

[#3]

Hello,

so there is no solution to install kafka 1.0.0 on HDP 2.6.4 and HDF 3.1.1?

if we create a HDF.repo and we install by hand "yum install kafka" and then "hdf-select set kafka-broker 3.1.1.0-35" and "su kafka -c" / usr / hdf / current / kafka-broker / bin / kafka start " ??

Or if we add HDF 3.1.1 on HDP 2.6.4 without a kafka service and then install kafka "yum install kafka" ?

Any suggestion please

[#4]

Hi @ranya triki

If you need Kafka 1.0 in HDP you will need to wait for HDP 2.6.5 coming in few weeks

[#5]

Hello @Abdelkrim Hadjidj

Thank you for your response

I upgraded to HDP 2.6.5 but i still have kafka 0.10.1 (Rolling Upgrade from HDP 2.5.3 to HDP 2.6.5)

I performed all the steps to have Kafka 1.0 as mention in this link:

https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.2.0/bk_ambari-upgrade/content/prepare-for-kafka...

Still have kafka 0.10.1 (Ambari UI > Admin > Stack and version > version Kafka)

Could you help me please

Thank you

[#6]

Hey @Abdelkrim Hadjidj

I've installed HDP 2.6.5, ambari tells me that I have Kafka 1.0.0 but edge says:

kafka_2_6_0_3_8-0.10.1.2.6.0.3-8.noarch

Could you confirm if this is version managment issue and how to really upgrade to kafka 1.0