Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
avatar
Super Collaborator

Spark Python Supportability Matrix

The Spark Python Supportability Matrix serves as an essential tool for determining which Python versions are compatible with specific Spark versions. This matrix provides a detailed overview of the compatibility levels for various Python versions across different Spark releases.

Spark Version Python Min Supported Version Python Max Supported Version

Python

v 2.7

Python

v3.4

Python

v3.5

Python

v 3.6

Python

v3.7

Python

v3.8

Python

v3.9

Python

v3.10

Python

v3.11

3.5.0 3.8 3.11 No No No No No Yes Yes Yes Yes
3.4.2 3.7 3.11 No No No No Yes Yes Yes Yes Yes
3.4.1 3.7 3.11 No No No No Yes Yes Yes Yes Yes
3.4.0 3.7 3.11 No No No No Yes Yes Yes Yes Yes
3.3.3 3.7 3.10 No No No No Yes Yes Yes Yes No
3.3.2 3.7 3.10 No No No No Yes Yes Yes Yes No
3.3.1 3.7 3.10 No No No No Yes Yes Yes Yes No
3.3.0 3.7 3.10 No No No No Yes Yes Yes Yes No
3.2.4 3.6 3.9 No No No Yes Yes Yes Yes No No
3.2.3 3.6 3.9 No No No Yes Yes Yes Yes No No
3.2.2 3.6 3.9 No No No Yes Yes Yes Yes No No
3.2.1 3.6 3.9 No No No Yes Yes Yes Yes No No
3.2.0 3.6 3.9 No No No Yes Yes Yes Yes No No
3.1.3 3.6 3.9 No No No Yes Yes Yes Yes No No
3.1.2 3.6 3.9 No No No Yes Yes Yes Yes No No
3.1.1 3.6 3.9 No No No Yes Yes Yes Yes No No
3.0.3 2.7/3.4 3.9 Yes Yes Yes Yes Yes Yes Yes No No
3.0.2 2.7/3.4 3.9 Yes Yes Yes Yes Yes Yes Yes No No
3.0.1 2.7/3.4 3.8 Yes Yes Yes Yes Yes Yes No No No
3.0.0 2.7/3.4 3.8 Yes Yes Yes Yes Yes Yes No No No
3.0.0 2.7/3.4 3.8 Yes Yes Yes Yes Yes Yes No No No
3.0.0 2.7/3.4 3.8 Yes Yes Yes Yes Yes Yes No No No
2.4.8 2.7/3.4 3.7 Yes Yes Yes Yes Yes No No No No
2.4.7 2.7/3.4 3.7 Yes Yes Yes Yes Yes No No No No
2.4.6 2.7/3.4 3.7 Yes Yes Yes Yes Yes No No No No
2.4.5 2.7/3.4 3.7 Yes Yes Yes Yes Yes No No No No
2.4.4 2.7/3.4 3.7 Yes Yes Yes Yes Yes No No No No
2.4.3 2.7/3.4 3.7 Yes Yes Yes Yes Yes No No No No
2.4.2 2.7/3.4 3.7 Yes Yes Yes Yes Yes No No No No
2.4.1 2.7/3.4 3.7 Yes Yes Yes Yes Yes No No No No
2.4.0 2.7/3.4 3.7 Yes Yes Yes Yes Yes No No No No
2.3.4 2.7/3.4 3.7 Yes Yes Yes Yes Yes No No No No
2.3.3 2.7/3.4 3.7 Yes Yes Yes Yes Yes No No No No
2.3.2 2.7/3.4 3.7 Yes Yes Yes Yes Yes No No No No
2.3.1 2.7/3.4 3.6 Yes Yes Yes Yes No No No No No
2.3.0 2.7/3.4 3.6 Yes Yes Yes Yes No No No No No
2.2.3 2.7/3.4 3.5 Yes Yes Yes No No No No No No
2.2.2 2.7/3.4 3.5 Yes Yes Yes No No No No No No
2.2.1 2.7/3.4 3.5 Yes Yes Yes No No No No No No
2.2.0 2.7/3.4 3.5 Yes Yes Yes No No No No No No
2.1.3 2.7/3.4 3.5 Yes Yes Yes No No No No No No
2.1.2 2.7/3.4 3.5 Yes Yes Yes No No No No No No
2.1.1 2.7/3.4 3.5 Yes Yes Yes No No No No No No
 
Note: The above data is generated using https://pypi.org/project/pyspark/ website. If you face any problems with supported python environment share in comments so that we can put some notes.
 
2,121 Views
Version history
Last update:
‎12-10-2023 09:50 PM
Updated by: