question Re: Spark 2.0 still TP? in Archives of Support Questions (Read Only)

Spark 2.0 still TP?

n3 — Fri, 17 Mar 2017 20:50:01 GMT

Hello,

I'm installing hdp and Ambari ask me to choose the services which I want to install. I see, that Spark 2.0 is still marked as Technical Preview. Is this correct?

I want to use it for my study, so sould I install Spark, Spark 2.0, or Spark+Spark2.0? I prefer Spark 2.0, but I'm not sure if it is stable enough and if not, can I install both at the same time without any problem?

Best regards

Re: Spark 2.0 still TP?

gmartin — Fri, 17 Mar 2017 20:56:44 GMT

With the latest version of HDP - 2.5.3, you can install Spark 1.6.2 (GA and stable), as well as Spark 2.0.1 (Tech Preview).

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_release-notes/content/ch_relnotes_v253.html

The install via Ambari, will create two separate Spark environments on the cluster.

You can access different Spark versions as detailed in:

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_spark-component-guide/content/spark-choose-version.html

Regards

Re: Spark 2.0 still TP?

rpathak — Fri, 17 Mar 2017 20:58:26 GMT

@Jan Horton

Yes, you can install both at the same time without any problem.

Spark 2.0 is TP as of HDP 2.5.3 and not recommended for production use, however you are welcome to use it for study.

Re: Spark 2.0 still TP?

n3 — Tue, 21 Mar 2017 22:08:51 GMT

Thank you very much. I installed both and it works fine.