Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Capacity Scheduler ordering policy is not reflecting

Capacity Scheduler ordering policy is not reflecting

Hi All,

I am using plain vanilla hadoop version 2.7.7. I have configured capacity scheduler order policy as "fair" in capacity-scheduler.xml but it is not reflecting in resource manager. Kindly help me in this.

capacity-scheduler.xml:

<configuration>
<property>
<name>yarn.scheduler.capacity.maximum-am-resource-percent</name>
<value>0.2</value>
</property>
<property>
<name>yarn.scheduler.capacity.maximum-applications</name>
<value>10000</value>
</property>
<property>
<name>yarn.scheduler.capacity.node-locality-delay</name>
<value>40</value>
</property>
<property>
<name>yarn.scheduler.capacity.queue-mappings-override.enable</name>
<value>false</value>
</property>
<property>
<name>yarn.scheduler.capacity.resource-calculator</name>
<value>org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.accessible-node-labels</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.acl_administer_queue</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.acl_submit_applications</name>
<value>*</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.maximum-capacity</name>
<value>100</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.ordering-policy</name>
<value>fair</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.ordering-policy.fair.enable-size-based-weight</name>
<value>false</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.priority</name>
<value>0</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.state</name>
<value>RUNNING</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.user-limit-factor</name>
<value>1</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.priority</name>
<value>0</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.queues</name>
<value>default</value>
</property>
</configuration>

Hadoopservice_0-1591593239752.png

@jsensharma @MattWho @nsabharwal @aervits @Shelton Need you expertise here. Pls advise

Regards,

Vignesh

4 REPLIES 4
Highlighted

Re: Capacity Scheduler ordering policy is not reflecting

Mentor

@Hadoopservice 

 

Sorry, I was away for a while is this still an issue?

Highlighted

Re: Capacity Scheduler ordering policy is not reflecting

Thanks for your reply. Yes, still i am facing this issue

Highlighted

Re: Capacity Scheduler ordering policy is not reflecting

Explorer

@Hadoopservice :

 

Did you try to refresh the yarn queue after the configuration. 

  1. In Ambari Web, browse to Services > YARN > Summary.
  2. Click Actions > Refresh YARN Capacity Scheduler.
  3. Confirm that you want to perform this operation. The refresh operation is submitted to the YARN ResourceManager.
Highlighted

Re: Capacity Scheduler ordering policy is not reflecting

Explorer
yarn rmadmin -refreshQueues

Try this command line if your are not using ambari. And let me know how this helps..

 

Try to update this while using yarn user. 

Don't have an account?
Coming from Hortonworks? Activate your account here