Support Questions
Find answers, ask questions, and share your expertise

setting vcores for yarn config and in spark-submit cmd

Hello,

 

if in yarn-site.xml of my resourcemanager node i specify the following: 

 

<property>
      <name>yarn.scheduler.minimum-allocation-vcores</name>
      <value>1</value>
</property>

<property>
      <name>yarn.scheduler.maximum-allocation-vcores</name>
      <value>3</value>
</property>

 and then submit to yarn cluster mode i specify 

 

--driver-cores 7

 

since i am running over a VM having 8 cores (32GB) will the above setting enable to successful submission of my spark application to the cluster ? why should be changed ? how about the memory setting ? is it the same ?

0 REPLIES 0