Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

need to manually upgrade spark from 2.3 to 2.4 within HDP 2.6.5

Highlighted

need to manually upgrade spark from 2.3 to 2.4 within HDP 2.6.5

New Contributor

Without having to do HDP full-stack upgrade, need to manually upgrade spark from 2.3 to 2.4

2 REPLIES 2

Re: need to manually upgrade spark from 2.3 to 2.4 within HDP 2.6.5

Cloudera Employee

Hi @vkatul,

Spark 2.4 is not officially tested, therefore it is not supported in HDP 2.6.5. If you still want to proceed with upgrading Spark version, you could find the procedure online.

Here is a list of supported component versions in HDP 2.6.5:

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.5/bk_release-notes/content/comp_versions.html

Re: need to manually upgrade spark from 2.3 to 2.4 within HDP 2.6.5

New Contributor

Actually, I did install spark 2.4 and set path on the Namenode2 where original Spark2.3 was residing - I can enter spark shell and verify it is 2.4, However, in Ambari Dashboard Services it still shows 2.3. Can you guide me to a weblink that can help?