Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

what is the range of spark version that CDH6.3.x supported?

Highlighted

what is the range of spark version that CDH6.3.x supported?

Explorer

what is the range of spark version that CDH6.3.x supported?

does CDH6.3.x support spark3.x?

5 REPLIES 5

Re: what is the range of spark version that CDH6.3.x supported?

Expert Contributor
Highlighted

Re: what is the range of spark version that CDH6.3.x supported?

Explorer

how about spark3.x and 1.6???   or CDH6.3 is only spark2.x.x supported

Highlighted

Re: what is the range of spark version that CDH6.3.x supported?

Expert Contributor

Spark1.6/2.3 arent available in CDH6.3 - Unlike CDH5 you cant add another spark version in CDH6

Quote from doc:

Important: CDH 6 does not support using multiple versions of Spark. Upgrading to CDH 6 disables the CDS 2 parcel if it exists, and all Spark services will use the built-in Spark version

https://docs.cloudera.com/documentation/enterprise/upgrade/topics/ug_spark_standalone.html

Highlighted

Re: what is the range of spark version that CDH6.3.x supported?

Explorer

thank you,now i know that CDH6.3.x is only support one spark。

my second question is:can i replace the build-in spark,for example,CDH6.3.3 's build-in spark version is 2.4.0, can i replace it to spark3.0 or spark 2.0

does CDH6.3 supported spark2 - 3 ???

Highlighted

Re: what is the range of spark version that CDH6.3.x supported?

Expert Contributor

Having the possibility to have different builds of Apache Spark in 6.x is an interesting question, however, the Apache Spark versions also should be compatible with 6.x.
The Apache Spark 2.2 and 2.3 are not compatible with 2.4.x, so this means the platform supporting 2.4.x may not work with 2.2.x. On the other hand, Apache Spark 3.x is on Scala 2.12 only, so I am interested to know if this possible on 6.x at all.

Don't have an account?
Coming from Hortonworks? Activate your account here