Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Spark 2.3 with Cloduera Version 6?

avatar
New Contributor

Hi,

I'm currently CDH 5.14 with Spark2, which uses the Spark CDS 2.3 (release 4).
I plan on upgrading to Cloudera Version 6, which I know uses CDS 2.2.  It's my understanding that before the upgrade I will need to remove Spark 2 via Cloduera Manager 

"If you are using a CDS 2 minor version higher than the version of Spark 2 included in the CDH 6 release you are upgrading to (2.2 in CDH 6.0.0), you must remove your Spark 2 services from Cloudera Manager."
https://www.cloudera.com/documentation/enterprise/upgrade/topics/ug_spark_standalone.html#concept_al...

As part of the upgrade, Spark2 CDS 2.2 would be installed.  At this point am I able to then re download the Service Descriptor for 2.3 (release 4) and upgrade to that?

1 ACCEPTED SOLUTION

avatar
Super Collaborator
Spark2 now comes bundled with CDH thus for CDH 6.0 you are tied to the Spark 2.2 release. You will not be able to install Spark 2.3 separately. Please be aware you are facing a transitional issue from separate Spark2 distribution to CDH distribution. Future CDH versions will update Spark2 releases accordingly to the latest versions, thus this situation would not appear anymore. If Spark 2.3 is a requirement we recommend to wait for the C6.1 release instead.

View solution in original post

11 REPLIES 11

avatar

asked and answered here.

 

 

Bill Brooks, Community Moderator
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
Explorer

can i replace the spark version from high to low? for example,cdh6.0 's build-in spark version is 2.2,can i replace it to 2.1