Support Questions

Find answers, ask questions, and share your expertise

Standalone Spark using Ambari

New Contributor

I'm looking for a way to install Spark in standalone mode using Ambari. For HDP 2.3 it was not possible: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_spark-quickstart/content/ch_introduction.... But looks like for HDP 2.4 it is possible: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_HDP_RelNotes/content/community_features.....

Could anyone, please, confirm it is really possible and clarify how to do it.

Thanks.

1 ACCEPTED SOLUTION

Mentor

You can do it but it is not supported by hortonworks, you'd need to follow Spark Standalone instructions. it defeats the purpose though as you lose benefit of Ambari provisioning, security features (no kerberos unless you run Spark on YARN) and no support from vendor. you will also have to package and deploy yourself, all that said here's the 2.0 guide https://spark.apache.org/docs/latest/spark-standalone.html

Our doc does not say it is possible, it says we do not support it.

View solution in original post

2 REPLIES 2

Mentor

You can do it but it is not supported by hortonworks, you'd need to follow Spark Standalone instructions. it defeats the purpose though as you lose benefit of Ambari provisioning, security features (no kerberos unless you run Spark on YARN) and no support from vendor. you will also have to package and deploy yourself, all that said here's the 2.0 guide https://spark.apache.org/docs/latest/spark-standalone.html

Our doc does not say it is possible, it says we do not support it.

New Contributor

I was confused by "community features", I thought there is an available prepackaged Spark standalone Ambari service and it is possible to integrate it with existing Ambari installation easily. But looks like all this stuff needs to be done manually..

Anyway now it seems to be much more clear to me, thank for the reply!

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.