Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark submit spark.yarn.submit.waitAppCompletion=false

Spark submit spark.yarn.submit.waitAppCompletion=false

Explorer

spark-submit and spark2-submit support parameter

--conf spark.yarn.submit.waitAppCompletion=false

for long running applications this is great to lower memory usage on the edge nodes

 

Is there a way to force this parameter for all Spark jobs submitted to the cluster?

 

(Env is Cloudera 5.13.3 with Spark on Yarn)

Don't have an account?
Coming from Hortonworks? Activate your account here