Skip to main content
RangaReddy
Gold ⭐️
Gold ⭐️
Dec 11, 2023 5:39 AM UTC

Spark Python Supportability Matrix

  • Oct 5, 2026 8:40 AM UTC
  • 2 replies
  • 116247 views
  • Translate All

Spark Python Supportability Matrix

The Spark Python Supportability Matrix serves as an essential tool for determining which Python versions are compatible with specific Spark versions. This matrix provides a detailed overview of the compatibility levels for various Python versions across different Spark releases.

Spark VersionPython Min Supported VersionPython Max Supported Version

Python

v 2.7

Python

v3.4

Python

v3.5

Python

v 3.6

Python

v3.7

Python

v3.8

Python

v3.9

Python

v3.10

Python

v3.11

3.5.23.83.11NoNoNoNoNoYesYesYesYes
3.5.13.83.11NoNoNoNoNoYesYesYesYes
3.5.03.83.11NoNoNoNoNoYesYesYesYes
3.4.23.73.11NoNoNoNoYesYesYesYesYes
3.4.13.73.11NoNoNoNoYesYesYesYesYes
3.4.03.73.11NoNoNoNoYesYesYesYesYes
3.3.33.73.10NoNoNoNoYesYesYesYesNo
3.3.23.73.10NoNoNoNoYesYesYesYesNo
3.3.13.73.10NoNoNoNoYesYesYesYesNo
3.3.03.73.10NoNoNoNoYesYesYesYesNo
3.2.43.63.9NoNoNoYesYesYesYesNoNo
3.2.33.63.9NoNoNoYesYesYesYesNoNo
3.2.23.63.9NoNoNoYesYesYesYesNoNo
3.2.13.63.9NoNoNoYesYesYesYesNoNo
3.2.03.63.9NoNoNoYesYesYesYesNoNo
3.1.33.63.9NoNoNoYesYesYesYesNoNo
3.1.23.63.9NoNoNoYesYesYesYesNoNo
3.1.13.63.9NoNoNoYesYesYesYesNoNo
3.0.32.7/3.43.9YesYesYesYesYesYesYesNoNo
3.0.22.7/3.43.9YesYesYesYesYesYesYesNoNo
3.0.12.7/3.43.8YesYesYesYesYesYesNoNoNo
3.0.02.7/3.43.8YesYesYesYesYesYesNoNoNo
3.0.02.7/3.43.8YesYesYesYesYesYesNoNoNo
3.0.02.7/3.43.8YesYesYesYesYesYesNoNoNo
2.4.82.7/3.43.7YesYesYesYesYesNoNoNoNo
2.4.72.7/3.43.7YesYesYesYesYesNoNoNoNo
2.4.62.7/3.43.7YesYesYesYesYesNoNoNoNo
2.4.52.7/3.43.7YesYesYesYesYesNoNoNoNo
2.4.42.7/3.43.7YesYesYesYesYesNoNoNoNo
2.4.32.7/3.43.7YesYesYesYesYesNoNoNoNo
2.4.22.7/3.43.7YesYesYesYesYesNoNoNoNo
2.4.12.7/3.43.7YesYesYesYesYesNoNoNoNo
2.4.02.7/3.43.7YesYesYesYesYesNoNoNoNo
2.3.42.7/3.43.7YesYesYesYesYesNoNoNoNo
2.3.32.7/3.43.7YesYesYesYesYesNoNoNoNo
2.3.22.7/3.43.7YesYesYesYesYesNoNoNoNo
2.3.12.7/3.43.6YesYesYesYesNoNoNoNoNo
2.3.02.7/3.43.6YesYesYesYesNoNoNoNoNo
2.2.32.7/3.43.5YesYesYesNoNoNoNoNoNo
2.2.22.7/3.43.5YesYesYesNoNoNoNoNoNo
2.2.12.7/3.43.5YesYesYesNoNoNoNoNoNo
2.2.02.7/3.43.5YesYesYesNoNoNoNoNoNo
2.1.32.7/3.43.5YesYesYesNoNoNoNoNoNo
2.1.22.7/3.43.5YesYesYesNoNoNoNoNoNo
2.1.12.7/3.43.5YesYesYesNoNoNoNoNoNo
 
Note: The above data is generated using https://pypi.org/project/pyspark/ website. If you face any problems with supported python environment share in comments so that we can put some notes.
 

2 replies

yarlanil91
Silver ⭐️
Jul 3, 2024 11:20 AM UTC

Hi,

Is anyone having issue while working on empty pyspark dataframe?

I installed spark 3.5.1 version and python 3.12.x, Having issue only on empty Dataframe.

spark 3.5.1 is working fine on python 3.11.x.

Any suggestion on python 3.12.x compatibility on spark 3.51?

 

 

Translate
Gopinath
Gold ⭐️⭐️
Gold ⭐️⭐️
Aug 19, 2024 5:11 AM UTC

Pyspark 3.5.2  - python >= 3.8 and <=3.11
ref: https://pypi.org/project/pyspark/3.5.2/

Translate