Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Is it possible to use Spark 2.3 with CDH 6.0.x or 6.1.x?

Highlighted

Is it possible to use Spark 2.3 with CDH 6.0.x or 6.1.x?

Explorer

I am running CDH and CM 6.1.x and trying to install Spark 2.3- however after distributing the parcel, when I choose "activate" I get the following message:

4 REPLIES 4
Highlighted

Re: Is it possible to use Spark 2.3 with CDH 6.0.x or 6.1.x?

Explorer

Actually, I see that the jar I am trying to use appears to be for cdh 5.13. Is there one for 6.x?

Highlighted

Re: Is it possible to use Spark 2.3 with CDH 6.0.x or 6.1.x?

Explorer

I was able to get the parcel activated, but now Spark2 does not display under "Add Services", even after restarting via the instructions.

Highlighted

Re: Is it possible to use Spark 2.3 with CDH 6.0.x or 6.1.x?

Cloudera Employee

 Spark 2.4 service is now (in C6.1) provided by the CDH parcel. There is no separate Spark2 parcel now. Please refer to the following link about Migrating Apache Spark Before Upgrading to CDH 6 :

https://docs.cloudera.com/documentation/enterprise/upgrade/topics/ug_spark_standalone.html#concept_p...

Highlighted

Re: Is it possible to use Spark 2.3 with CDH 6.0.x or 6.1.x?

Cloudera Employee

Hi, 

 

CDH 6.1 package comes with spark 2.4 version. please see the below link for more information

 

https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_61_packaging.html

 

CDH 6 does not have separated parcels for Spark. it comes with CDH
 
Thanks
AK
Don't have an account?
Coming from Hortonworks? Activate your account here