Hi, I have Ambari cluster setup and submitting Spark jobs using spark-submit in YARN Client mode. When I go to YARN UI, I see some of the jobs but not others. note, all the jobs ran and completed successfully. When I look at the output of Spark-submit, jobs which are showing up in YARN UI have application ID printed and rest don't have that line in the log.   In the spark-submit output I can see that both jobs have spawned executors.  Why are some jobs don't have application ID printed and not showing up in YARN UI ? What needs to be done. I can't use YARN Cluster mode, as in that case Spark-submit command will not show log of job progress .   ... View more