Latest Python3.x version compatible with HDP3.1.5

[#1]

Hi,

We are currently using Python2.7 with HDP3.1.5 (PySpark) and RHEL 7.9.

We are planning to upgrade to Python3.x as Python2.7 support has ended.

If anyone could please help determining the latest stable Python3.x? version that will best suit or compatible with HDP3.1.5 (Spark2.3) and RHEL 7.9?

OR

Let me know steps I should follow to determine the best stable Python3.x? version for the HDP stack?

Thanks,

PB

[#2]

Hi Priyabrat,

Just for your reference.
From CDP Private Cloud doc, it stated some Spark and Python version support pairs.
https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/release-guide/topics/cdpdc-os-requirement...

I think Spark 2.3 should share similar Python support as Spark 2.4.

For compatibility issues, I think it is better to raise support case to Cloudera support team.

[#3]

Hi @Priyabrat 

Please refer below Doc[1], Ambari 2.7.5 supports only HDP-3.1.5 and HDF-3.2.0 and for Ambari 2.7.5 the software requirements are mentioned in doc[2]

[1]https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/bk_ambari-installation/content/determine_produ...

[2]https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/bk_ambari-installation/content/mmsr_software_r...

[#4 · Accepted solution]

Seems the max. Python version supported is 2.7.5, Python Version 3.x is not supported as it will create conflict with the versions shipped (HDP3.1.5 + Ambari2.7.5).

Python3 support is not on the road map of HDP 3x and even for Ambari. This is something which may be done in future to make Python3 as the default for the OS Cloudera supports.