Support Questions
Find answers, ask questions, and share your expertise

Need help on Spark Upgrade

Solved Go to solution
Highlighted

Re: Need help on Spark Upgrade

once upgrade completed we also need to clicked on finalize button to complete the upgrade process. if you have done this and still see previous version of spark then you should login to node and manually check spark package version through "rpm -qa|grep spark" command.

Highlighted

Re: Need help on Spark Upgrade

Expert Contributor
@Jitendra Yadav

..thanks for your suggestion. Me being a newbie to this system : Can I upgrade through Ambari ?

Re: Need help on Spark Upgrade

Agreed, you should at least upgrade the lower HDP version (...2.3.0...) to the newer HDP version (2.3.4.0-3485). It is best to get the default Spark version from the HDP install. Please see Table 1.1 at this link which describes the version associations for HDP, Ambari, and Spark:

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_spark-guide/content/ch_introduction-spark...