How to upgrade to spark 2?

[#1]

I am using spark 1.4.1 HDP 2.3.2. How do I upgrade to spark 2.0?

[#2 · Accepted solution]

@Eon kitex

Hortonworks does a lot of integration testing before it releases any new version of HDP. It means that Hortons makes sure that all components being part of HDP have been tested to work together so that they are supportable and enterprise ready. You cannot upgrade individual components. If you want to just test spark 2.0, you can download it separately on a different machine and evaluate it. But you cannot upgrade just spark HDP 2.3.2.

When Hortonworks starts supporting Spark 2.0, it will be included in newer versions of HDP.

[#3]

Spark 2.0 is coming as technical preview on HDP 2.5. You can also try it now with Hortonworks cloud, See this blog

[#4]

Is there any date when hdp update will be released? Is there anyway I could manually update spark?

[#5]

@Eon kitex

I cannot speak on behalf of product team. If you are a Hortonworks customer, this is something you can discuss with your account team. I am sure, they will be able to answer your questions.

[#6]

Spark 2.0.0 will be available with HDP-2.5

https://community.hortonworks.com/articles/53029/how-to-install-and-run-spark-20-on-hdp-25-sandbox.h....