Is it possible to have two versions of Kafka in the same CDH cluster ?

[#1]

We already have a Kafka in our CDH cluster and we wish to set up a newer version of Kafka in the same cluster. I have tried adding new Kafka parcel in CM and it seems only one version of Kafka parcel can be enabled at the same time. I wonder whether it is possible. Thanks.

[#2 · Accepted solution]

Hi @iamabug 

This is not possible unfortunately. You could only have one parcel for a specific service activated at any given time. You would need to have a separate cluster managed by a different Cloudera Manager installation in order to activate a different Kafka parcel version.

[#3]

Thanks. Do you mean two clusters managed by one CM or two clusters managed by two different CMs ? I believe it is the former.

[#4]

Two different clusters managed by two different Cloudera Managers.

[#5]

I managed to install a second cluster, which only includes Kafak and Zookeeper, as shown in the figure below. w@leed 

[#6]

Hi @iamabug 

Yes, you can definitely have two different clusters managed by the same Cloudera Manager instance. In return, you can have different CDH versions for the two clusters, however, my initial thoughts was that you wouldn't be able to activate two different Kafka parcel versions, for example CDK 3.0 on Cluster A and CDK 2.2 on Cluster B.

But I need to double check this because I have some doubts if that is in fact the case in CDH 5.x or not. In CDH 6.x you're bound to a specific version of Kafka depending on your CDH version since that's not released as a separate customer service so my response was geared more towards C6 rather than C5.

Do you have two different versions of Kafka running on cluster 1 and cluster 2? 

[#7]

I have CDK 3.1.0 running on cluster 1 and CDK 4.1.0 running on cluster 2. And CDH version is 5.15.1.

[#8]

Thanks for confirming.

Yes, I see that this is in fact possible in CDH 5 as well when I did a quick check. 

Just remember that starting with CDH 6.0, both Spark 2.x and Kafka are bundled in CDH, so the only way you would have two different versions is by running two different CDH versions from Cloudera Manager.

[#9]

Now I am really clear about the situation. Thanks a lot for your replies.