Skip to main content
javidshaik
javidshaik
Silver ⭐️⭐️
June 27, 2021
Solved

upgrading spark to 2.4

Dear All,

 

On one particular cluster we have spark 2.3 running(CDH 5.15.1). can it be possible to only upgrade spark to 2.4 ? by remaining on the same CDH version(5.15.1). will it compatible with existing service ?. kindly suggest  

RangaReddy
RangaReddyAnswer
Gold ⭐️
June 29, 2021

Hi @javidshaik 

 

Yes based on cloudera documentation, it is not supported multiple versions under same Cloudera Manager Server.

View in thread

12 replies

RangaReddy
Gold ⭐️
RangaReddy
Gold ⭐️
June 28, 2021

Hi @javidshaik 

 

CDH 5.x and HDP 2.X version clusters has reached end of life support. Better upgrade your cluster to CDH 6.X/CDP 7.X version.

 

Both CDH 6.X and CDP 7.X clusters will support Spark 2.4 versions.

 

Please refer following documentation:

https://www.cloudera.com/legal/policies/support-lifecycle-policy.html

 

Translate
javidshaik
javidshaikAuthor
Silver ⭐️⭐️
June 28, 2021

Hi @RangaReddy

 

Yeah true i agree. thanks, but just wanted to know is it feasible or not to only update to spark2.4 on the cluster  ? or will there be any compatibility issues.  Appreciate your response 

Translate
javidshaik
javidshaikAuthor
Silver ⭐️⭐️
June 28, 2021

hi @RangaReddy 

 

thanks for the below details.

 

Basically i have two(PROD & Stage) CDH (5.15.1) clusters with spark 2.3(2.3.0.cloudera4-1.cdh5.13.3.p0.611179) running on each cluster  under one CM. so i wanted to upgrade to spark2.4 only on one(stage) CDH(5.15.1) cluster. so as per below documentation link provided its being said

 

" All CDH clusters managed by the same Cloudera Manager Server must use exactly the same version of CDS Powered by Apache Spark. For example, you cannot use the built-in CDH Spark service, a CDS 2.1 service, and a CDS 2.2 service. You must choose only one CDS 2 Powered by Apache Spark release"

 

So i believe we cannot perform upgrade on this right. kindly confirm 

Translate
Terms & ConditionsAccessibility statement