Hi Priyabrat,   Just for your reference. From CDP Private Cloud doc, it stated some Spark and Python version support pairs. https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/release-guide/topics/cdpdc-os-requirements.html   I think Spark 2.3 should share similar Python support as Spark 2.4. For compatibility issues, I think it is better to raise support case to Cloudera support team.   ... View more