https://community.cloudera.com/t5/Support-Questions/I-am-unable-to-run-a-spark-job-successfully-using-Yarn-Rest/m-p/230946#M192793 <P><A rel="user" href="https://community.cloudera.com/users/11608/ssharma.html" nodeid="11608">@ssharma</A> </P><P>Following is the command I use to submit the spark job :</P><P>{{JAVA_HOME}}/bin/java -Xmx1024m org.apache.spark.deploy.yarn.ApplicationMaster --class 'org.apache.spark.examples.SparkPi' --jar AppMaster.jar 1&gt;&lt;LOG_DIR&gt;/AppMaster.stdout 2&gt;&lt;LOG_DIR&gt;/AppMaster.stderr.</P><P><STRONG>I have mentioned the payload in the question only where I have used this command.</STRONG></P><P>Although when I use the spark-submit (from the payload) I am able to run the spark job successfully. But I think this is a hack as it submits two different jobs, first one because of my yarn rest API and then internally the second one because of the spark command which is mentioned in the payload. Following is the spark submit command :</P><P>{{SPARK_HOME}}/bin/spark-submit --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode cluster --driver-memory 1g --executor-memory 1g --executor-cores 1 /tmp/spark-examples_2.11-2.1.0.2.6.0.3-8.jar 10</P><P>Hence I wanted to run a job successfully using java command.</P> Wed, 20 Jun 2018 12:10:20 GMT abhinavkumar219 2018-06-20T12:10:20Z