Skip to main content
javidshaik
javidshaik
Silver ⭐️⭐️
June 27, 2021
Solved

upgrading spark to 2.4

Dear All,

 

On one particular cluster we have spark 2.3 running(CDH 5.15.1). can it be possible to only upgrade spark to 2.4 ? by remaining on the same CDH version(5.15.1). will it compatible with existing service ?. kindly suggest  

RangaReddy
RangaReddyAnswer
Gold ⭐️
June 29, 2021

Hi @javidshaik 

 

Yes based on cloudera documentation, it is not supported multiple versions under same Cloudera Manager Server.

View in thread

12 replies

RangaReddy
Gold ⭐️
RangaReddy
Gold ⭐️
June 28, 2021

Hi @javidshaik 

 

CDH 5.x and HDP 2.X version clusters has reached end of life support. Better upgrade your cluster to CDH 6.X/CDP 7.X version.

 

Both CDH 6.X and CDP 7.X clusters will support Spark 2.4 versions.

 

Please refer following documentation:

https://www.cloudera.com/legal/policies/support-lifecycle-policy.html

 

Translate
javidshaik
javidshaikAuthor
Silver ⭐️⭐️
June 28, 2021

Hi @RangaReddy

 

Yeah true i agree. thanks, but just wanted to know is it feasible or not to only update to spark2.4 on the cluster  ? or will there be any compatibility issues.  Appreciate your response 

Translate
Rajeshhadoop
Rajeshhadoop
Silver ⭐️⭐️⭐️
February 18, 2022

Hi 

 

As mentioned below, Spark 2.3 -> 2.4 version changes have higher potential of risks.

May i know what are the those risks. It would be more helpful if you share right link for these and also share what the considerations need to be taken while upgrading form spark 1.6 to 2.4 and spark 2.3 to spark 2.4

 

Thanks

Rajesh

Translate
Terms & ConditionsAccessibility statement