Support Questions
Find answers, ask questions, and share your expertise

configure spark standalone using ambari


currently i have spark over yarn configured and working (on 9 servers approx 200 CORES)

i would like to configure them in standalone mode ( to prevent the "waist of time of allocation containers" )

is it possible ?





One more time, someone one can help?






You can set your deployment mode in configuration files or from the command line when submitting a job. Use one of the following options to set the deployment mode:

  • Client deployment mode.                     [The driver runs locally]
  • Cluster deployment mode.                  [The driver runs on the cluster]

Standalone Cluster Mode
As the name suggests, its a standalone cluster with only spark specific components. It doesn’t have any dependencies on Hadoop components and Spark driver acts as the cluster manager.

To launch a Spark application in cluster mode:

$ ./bin/spark-submit --class --master yarn --deploy-mode cluster [options] <app jar> [app options]


Single node

To launch a Spark application in client mode, do the same, but replace cluster with the client. The following shows how you can run spark-shell in client mode:

$ ./bin/spark-shell --master yarn --deploy-mode client


Hope that helps


Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.