Support Questions
Find answers, ask questions, and share your expertise
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

Kafka on HDP & HDF

New Contributor

We want to install Kafka on our HDP cluster. We have installed HDF management pack on ambari server, but still it only shows NiFI and NiFI Registry, Kafka version shown under "Add services" is older i.e. 0.10

Any suggestion how to add Kafka on HDP ? @Jay Kumar SenSharma


Hi @Ankur Gupta

Using the same Ambari for HDF and HDP have some caveat with the latest version.

You cannot install SAM and Schema Registry for HDF 3.1 on an HDP 2.6.4 cluster, and you cannot upgrade your Storm and Kafka versions if they exist on an HDP cluster. This is a temporary limitation with HDF 3.1 and will be resolved with HDP 3.0 and HDF 3.2. This has been described in the doc

To have Kafka 1.x in HDP you need to wait for HDP 3.0

New Contributor


so there is no solution to install kafka 1.0.0 on HDP 2.6.4 and HDF 3.1.1?

if we create a HDF.repo and we install by hand "yum install kafka" and then "hdf-select set kafka-broker" and "su kafka -c" / usr / hdf / current / kafka-broker / bin / kafka start " ??

Or if we add HDF 3.1.1 on HDP 2.6.4 without a kafka service and then install kafka "yum install kafka" ?

Any suggestion please

Hi @ranya triki

If you need Kafka 1.0 in HDP you will need to wait for HDP 2.6.5 coming in few weeks

New Contributor

Hello @Abdelkrim Hadjidj

Thank you for your response

I upgraded to HDP 2.6.5 but i still have kafka 0.10.1 (Rolling Upgrade from HDP 2.5.3 to HDP 2.6.5)

I performed all the steps to have Kafka 1.0 as mention in this link:

Still have kafka 0.10.1 (Ambari UI > Admin > Stack and version > version Kafka)

Could you help me please

Thank you

New Contributor

Hey @Abdelkrim Hadjidj

I've installed HDP 2.6.5, ambari tells me that I have Kafka 1.0.0 but edge says:


Could you confirm if this is version managment issue and how to really upgrade to kafka 1.0