Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Is it possible to have two versions of Kafka in the same CDH cluster ?

Solved Go to solution
Highlighted

Is it possible to have two versions of Kafka in the same CDH cluster ?

Explorer

We already have a Kafka in our CDH cluster and we wish to set up a newer version of Kafka in the same cluster. I have tried adding new Kafka parcel in CM and it seems only one version of Kafka parcel can be enabled at the same time. I wonder whether it is possible. Thanks.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Is it possible to have two versions of Kafka in the same CDH cluster ?

Cloudera Employee

Hi @iamabug 

This is not possible unfortunately. You could only have one parcel for a specific service activated at any given time. You would need to have a separate cluster managed by a different Cloudera Manager installation in order to activate a different Kafka parcel version.


8 REPLIES 8

Re: Is it possible to have two versions of Kafka in the same CDH cluster ?

Cloudera Employee

Hi @iamabug 

This is not possible unfortunately. You could only have one parcel for a specific service activated at any given time. You would need to have a separate cluster managed by a different Cloudera Manager installation in order to activate a different Kafka parcel version.


Re: Is it possible to have two versions of Kafka in the same CDH cluster ?

Explorer

Thanks. Do you mean two clusters managed by one CM or two clusters managed by two different CMs ? I believe it is the former.

Re: Is it possible to have two versions of Kafka in the same CDH cluster ?

Cloudera Employee

Two different clusters managed by two different Cloudera Managers.

Re: Is it possible to have two versions of Kafka in the same CDH cluster ?

Explorer

I managed to install a second cluster, which only includes Kafak and Zookeeper, as shown in the figure below. w@leed 

屏幕快照 2019-08-21 下午3.15.35.png

Re: Is it possible to have two versions of Kafka in the same CDH cluster ?

Cloudera Employee

Hi @iamabug 

Yes, you can definitely have two different clusters managed by the same Cloudera Manager instance. In return, you can have different CDH versions for the two clusters, however, my initial thoughts was that you wouldn't be able to activate two different Kafka parcel versions, for example CDK 3.0 on Cluster A and CDK 2.2 on Cluster B.

But I need to double check this because I have some doubts if that is in fact the case in CDH 5.x or not. In CDH 6.x you're bound to a specific version of Kafka depending on your CDH version since that's not released as a separate customer service so my response was geared more towards C6 rather than C5.

Do you have two different versions of Kafka running on cluster 1 and cluster 2? 

Re: Is it possible to have two versions of Kafka in the same CDH cluster ?

Explorer

I have CDK 3.1.0 running on cluster 1 and CDK 4.1.0 running on cluster 2. And CDH version is 5.15.1.

Re: Is it possible to have two versions of Kafka in the same CDH cluster ?

Cloudera Employee

Thanks for confirming.

Yes, I see that this is in fact possible in CDH 5 as well when I did a quick check. 

Just remember that starting with CDH 6.0, both Spark 2.x and Kafka are bundled in CDH, so the only way you would have two different versions is by running two different CDH versions from Cloudera Manager.

Re: Is it possible to have two versions of Kafka in the same CDH cluster ?

Explorer

Now I am really clear about the situation. Thanks a lot for your replies.