question Hortonworks and Spark 2.0 in Archives of Support Questions (Read Only)

Hortonworks and Spark 2.0

marko_kole — Tue, 23 Feb 2016 17:12:30 GMT

According to the Databricks people at the Spark Summit 2016 in New York, Spark 2.0 is coming out in April/May this year. This means a "jump" from 1.6.0 to 2.0.

Spark 1.5.2 is installed with the latest Hortonworks distribution. Spark 1.6.0 is in technical preview, Spark 2.0 is around the corner. Im wondering what plans Hortonworks has around this matter?

Re: Hortonworks and Spark 2.0

pminovic — Tue, 23 Feb 2016 17:31:11 GMT

Hi @marko, if I remember correctly Spark-1.6 TP for HDP-2.3.4 was announced 1 day after its GA. Therefore, you can expect Spark-2.0 TP with about the same "delay", and its inclusion in one of the coming versions of HDP soon after. Stay tuned!

Re: Hortonworks and Spark 2.0

nsabharwal — Tue, 23 Feb 2016 17:39:39 GMT

@vshukla

Re: Hortonworks and Spark 2.0

aervits — Tue, 23 Feb 2016 18:46:21 GMT

No plans have been announced but we can guess around the same time as Apache stable release, so maybe when 2.0.1 lands.

Re: Hortonworks and Spark 2.0

rhryniewicz — Thu, 03 Mar 2016 10:46:27 GMT

With HDP 2.4 release, we now support Spark 1.6.

Re: Hortonworks and Spark 2.0

bigdatacn — Sat, 02 Jul 2016 20:41:20 GMT

any guides to upgrade spark from 1.6.1 to 1.6.2 we are on the HDP 2.4.

Re: Hortonworks and Spark 2.0

andrew11 — Fri, 29 Jul 2016 14:54:47 GMT

I'd like to upgrade my hortonworks 2.4.0.0-169 cluster to run Spark 2.0 now on GA to trial it, How to upgrade?