question Standalone Spark using Ambari in Archives of Support Questions (Read Only)

Standalone Spark using Ambari

azotcsit — Sat, 20 Aug 2016 03:11:19 GMT

I'm looking for a way to install Spark in standalone mode using Ambari. For HDP 2.3 it was not possible: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_spark-quickstart/content/ch_introduction-spark-quickstart.html. But looks like for HDP 2.4 it is possible: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_HDP_RelNotes/content/community_features.html.

Could anyone, please, confirm it is really possible and clarify how to do it.

Thanks.

Re: Standalone Spark using Ambari

aervits — Sat, 20 Aug 2016 21:14:39 GMT

You can do it but it is not supported by hortonworks, you'd need to follow Spark Standalone instructions. it defeats the purpose though as you lose benefit of Ambari provisioning, security features (no kerberos unless you run Spark on YARN) and no support from vendor. you will also have to package and deploy yourself, all that said here's the 2.0 guide https://spark.apache.org/docs/latest/spark-standalone.html

Our doc does not say it is possible, it says we do not support it.

Re: Standalone Spark using Ambari

azotcsit — Mon, 22 Aug 2016 23:27:41 GMT

I was confused by "community features", I thought there is an available prepackaged Spark standalone Ambari service and it is possible to integrate it with existing Ambari installation easily. But looks like all this stuff needs to be done manually..

Anyway now it seems to be much more clear to me, thank for the reply!