question spark 2 and spark 3 on cdp in Support Questions

spark 2 and spark 3 on cdp

clouderaskme — Tue, 21 Apr 2026 07:56:52 GMT

hello all,

I am currently running a python version of 3.6.8, Can I still install spark 3 on my cdp cluster?

Re: spark 2 and spark 3 on cdp

Bharati — Thu, 28 Apr 2022 18:31:15 GMT

@clouderaskme Please review the documents below that provides the details on requirements for Spark3.2, 3.1 and 3.0
https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/cds-3/topics/spark-3-requirements.html

https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/cds-3/topics/spark-spark-3-requirements.html

https://docs.cloudera.com/cdp-private-cloud-base/7.1.4/cds-3/topics/spark-spark-3-requirements.html

Cloudera Distributed Spark 3.2 requires Python 3.6+ and requires CDP 7.1.7 and higher
Cloudera Distributed Spark 3.1 requires Python 3.6+ and requires CDP 7.1.7 and higher
Cloudera Distributed Spark 3.0 requires Python 3.4 or higher. and requires CDP7.1.3, 7.1.4 and 7.1.5

Re: spark 2 and spark 3 on cdp

clouderaskme — Thu, 28 Apr 2022 18:46:38 GMT

but If i want to continue having spark 2 on it, until what version can python updated?

Re: spark 2 and spark 3 on cdp

Bharati — Thu, 28 Apr 2022 20:55:44 GMT

@clouderaskme The latest CDP 7.1.7 comes with the default Spark 2.4 version.

https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/runtime-release-notes/topics/rt-pvc-runtime-component-versions.html

Spark 2.4 supports Python 2.7 and 3.4-3.7.

https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/release-guide/topics/cdpdc-os-requirements.html