https://community.cloudera.com/t5/Support-Questions/SPARK-SUBMIT-issue-with-SPARK-2-2/m-p/217628#M179537 <A rel="user" href="https://community.cloudera.com/users/16922/vkc.html" nodeid="16922">@Vinod K C</A><P> I haven't come across any document but from the HDP installation you can find it from:<STRONG> </STRONG><STRONG>/etc/spark2/conf/spark-env.sh</STRONG></P><PRE># Options read in YARN client mode #SPARK_EXECUTOR_INSTANCES="2" #Number of workers to start (Default: 2) #SPARK_EXECUTOR_CORES="1" #Number of cores for the workers (Default: 1). #SPARK_EXECUTOR_MEMORY="1G" #Memory per Worker (e.g. 1000M, 2G) (Default: 1G) #SPARK_DRIVER_MEMORY="512M" #Memory for Master (e.g. 1000M, 2G) (Default: 512 Mb) #SPARK_YARN_APP_NAME="spark" #The name of your application (Default: Spark) #SPARK_YARN_QUEUE="default" #The hadoop queue to use for allocation requests (Default: default) #SPARK_YARN_DIST_FILES="" #Comma separated list of files to be distributed with the job. #SPARK_YARN_DIST_ARCHIVES="" #Comma separated list of archives to be distributed with the job.</PRE><P>But this says only YARN CLIENT mode.</P><P>And the job is not picking up the files available in /etc/spark2/conf as well.</P><P>Thanks</P><P>Venkat</P> Tue, 17 Apr 2018 21:39:09 GMT dheer_vijji_rag 2018-04-17T21:39:09Z