Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Spark version supported in Hortonworks 2.4.x

avatar
Contributor

Hi,

Is the version 2.0.0 of Spark supported in Hortonworks 2.4.x? Currently we have Spark 1.6 installed but we would need to update to 2.0.

Thanks a lot in advance.

1 ACCEPTED SOLUTION

avatar
Super Guru

Hi @Jaime

Currently HDP 2.4.2 only support spark 1.6, therefore you won't see spark 2.0 in the repository. We don't recommend to install versions which are not tested which specific HDP release.

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_HDP_RelNotes/content/ch_relnotes_v242.ht...

However if you are looking to test spark 2.0 on HDP then can you download the spark source code and build it with HDP dependencies.

View solution in original post

4 REPLIES 4

avatar
Super Guru

Hi @Jaime

Currently HDP 2.4.2 only support spark 1.6, therefore you won't see spark 2.0 in the repository. We don't recommend to install versions which are not tested which specific HDP release.

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_HDP_RelNotes/content/ch_relnotes_v242.ht...

However if you are looking to test spark 2.0 on HDP then can you download the spark source code and build it with HDP dependencies.

avatar

Hi @Jaime,

Spark 2.0 is not supported in 2.4.x.

Have a look here: https://hortonworks.com/blog/welcome-apache-spark-2-0/

Hope this helps.

avatar

Spark 2.0 is technical preview in HDP 2.5. You can try out 2.0 in Hortonworks cloud today or look for HDP 2.5 announcement to try it in your own env.

avatar
Contributor

Thanks @vshukla.