Standalone Spark using Ambari

[#1]

I'm looking for a way to install Spark in standalone mode using Ambari. For HDP 2.3 it was not possible: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_spark-quickstart/content/ch_introduction.... But looks like for HDP 2.4 it is possible: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_HDP_RelNotes/content/community_features.....

Could anyone, please, confirm it is really possible and clarify how to do it.

Thanks.

[#2 · Accepted solution]

You can do it but it is not supported by hortonworks, you'd need to follow Spark Standalone instructions. it defeats the purpose though as you lose benefit of Ambari provisioning, security features (no kerberos unless you run Spark on YARN) and no support from vendor. you will also have to package and deploy yourself, all that said here's the 2.0 guide https://spark.apache.org/docs/latest/spark-standalone.html

Our doc does not say it is possible, it says we do not support it.

[#3]

I was confused by "community features", I thought there is an available prepackaged Spark standalone Ambari service and it is possible to integrate it with existing Ambari installation easily. But looks like all this stuff needs to be done manually..

Anyway now it seems to be much more clear to me, thank for the reply!