Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark 2.4 on CDH

Spark 2.4 on CDH

New Contributor

Hi,

 

I'm currently running a cluster with Cloudera Enterprise 5.16.1 and I've upgraded it to use Spark 2.2. All is working as expected.

 

I now realise there's a feature in Spark 2.4 I'm keen to use but can't see an easy way to update my cluster since the latest CDS version available is 2.3 (https://www.cloudera.com/documentation/spark2/latest/topics/spark2_packaging.html#versions)

 

Is there an alternative way to get Spark 2.4 installed on the cluster or am I missing something obvious?

 

Thanks

1 REPLY 1

Re: Spark 2.4 on CDH

Guru
No, you can't upgrade single component individually outside of CDH, it is unsupported.

Which specific feature that you are after? It might be included in CDH, as the version in CDH and upstream are different, we backport lots of fixes into our version, even though it is behind the version number compared with upstream.