Support Questions
Find answers, ask questions, and share your expertise

YARN Setting User Limits Doc Inconsistency

New Contributor

In https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_yarn-resource-management/content/setting..., it says if second job is submitted it shares resources equally 50% with first job.

But that happens only in case of fair scheduling right ?

In case of FIFO, if first job needs 100%+ resources, it will be given 100% and even if second job is submitted, first job still takes up 100% resources and second job will get resources only after first job is completed. Isn't it ? In other case of first job taking up x resources (< 100%), then second job will take 100-x (but need not be 50 again). Isn't it ?

But the doc doesn't say even in FIFO, 50% sharing happens. How ?

2 REPLIES 2

Re: YARN Setting User Limits Doc Inconsistency

Explorer

Thanks @Rajesh KSV, I'll update the docs to point out that this example is for fair scheduling. FIFO vs fair scheduling is better explained in the Using Flexible Scheduling Policies topic, so I'll also add a link to that topic from the User Limits topic.

Re: YARN Setting User Limits Doc Inconsistency

New Contributor

Strangely, even the official docs say the same thing - http://hadoop.apache.org/docs/r2.6.0/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html . 2.6.0 doesn't even have fair ordering policy. Still it says resources are distributed equally among jobs (even when its FIFO).

Can you please add here - how will that param affects scheduling ?