spark 2 and spark 3 on cdp

[#1]

hello all,

I am currently running a python version of 3.6.8, Can I still install spark 3 on my cdp cluster?

[#2 · Accepted solution]

@clouderaskme Please review the documents below that provides the details on requirements for Spark3.2, 3.1 and 3.0
https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/cds-3/topics/spark-3-requirements.html

https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/cds-3/topics/spark-spark-3-requirements.html

https://docs.cloudera.com/cdp-private-cloud-base/7.1.4/cds-3/topics/spark-spark-3-requirements.html

Cloudera Distributed Spark 3.2 requires Python 3.6+ and requires CDP 7.1.7 and higher
Cloudera Distributed Spark 3.1 requires Python 3.6+ and requires CDP 7.1.7 and higher
Cloudera Distributed Spark 3.0 requires Python 3.4 or higher.  and requires CDP7.1.3, 7.1.4 and 7.1.5

[#3]

but If i want to continue having spark 2 on it, until what version can python updated?

[#4]

@clouderaskme  The latest CDP 7.1.7 comes with the default Spark 2.4 version. 

https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/runtime-release-notes/topics/rt-pvc-runtime-c...

Spark 2.4 supports Python 2.7 and 3.4-3.7.

https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/release-guide/topics/cdpdc-os-requirement...