Skip to main content
Priyabrat
Priyabrat
Gold ⭐️
August 23, 2021
Solved

Latest Python3.x version compatible with HDP3.1.5

Hi,

We are currently using Python2.7 with HDP3.1.5 (PySpark) and RHEL 7.9.

We are planning to upgrade to Python3.x as Python2.7 support has ended.

If anyone could please help determining the latest stable Python3.x? version that will best suit or compatible with HDP3.1.5 (Spark2.3) and RHEL 7.9?

OR

Let me know steps I should follow to determine the best stable Python3.x? version for the HDP stack?

 

Thanks,

PB

 

Priyabrat
PriyabratAnswer
Gold ⭐️
August 26, 2021

Seems the max. Python version supported is 2.7.5, Python Version 3.x is not supported as it will create conflict with the versions shipped (HDP3.1.5 + Ambari2.7.5).

Python3 support is not on the road map of HDP 3x and even for Ambari. This is something which may be done in future to make Python3 as the default for the OS Cloudera supports.

View in thread

3 replies

Yam
Yam
Silver ⭐️
August 24, 2021

Hi Priyabrat,

 

Just for your reference.
From CDP Private Cloud doc, it stated some Spark and Python version support pairs.
https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/release-guide/topics/cdpdc-os-requirements.html

 

I think Spark 2.3 should share similar Python support as Spark 2.4.

For compatibility issues, I think it is better to raise support case to Cloudera support team.

 

Translate
    jAnshula
    Gold ⭐️
    jAnshula
    Gold ⭐️
    August 24, 2021

    Hi @Priyabrat 

     

    Please refer below Doc[1], Ambari 2.7.5 supports only HDP-3.1.5 and HDF-3.2.0 and for Ambari 2.7.5 the software requirements are mentioned in doc[2]

    [1]https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/bk_ambari-installation/content/determine_product_interop.html

     

    [2]https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/bk_ambari-installation/content/mmsr_software_reqs.html

    Translate
      Priyabrat
      PriyabratAuthorAnswer
      Gold ⭐️
      August 26, 2021

      Seems the max. Python version supported is 2.7.5, Python Version 3.x is not supported as it will create conflict with the versions shipped (HDP3.1.5 + Ambari2.7.5).

      Python3 support is not on the road map of HDP 3x and even for Ambari. This is something which may be done in future to make Python3 as the default for the OS Cloudera supports.

      Translate
      Terms & ConditionsAccessibility statement