Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

setting vcores for yarn config and spark-submit

Hello,

 

if in yarn-site of my resourcemanager node i specify the following: 

 

<property>
      <name>yarn.scheduler.minimum-allocation-vcores</name>
      <value>1</value>
</property>

<property>
      <name>yarn.scheduler.maximum-allocation-vcores</name>
      <value>3</value>
</property>

 and then submit to yarn cluster mode i specify 

 

--driver-cores 7

 

since i am running over a VM having 8 cores (32GB) will the above setting enable to successful submission of my spark application to the cluster ? why should be changed ? how about the memory setting ? is it the same ?

0 REPLIES 0