Member since
04-21-2020
2
Posts
0
Kudos Received
0
Solutions
04-25-2020
08:46 AM
Hi Team, Please provide your inputs: This is would ve very helpful We have 3 queues, Q1, Q2, Q2 Q1: min-55% max 80% Q2: min-35% max 70% Q3: min-10% max 90% We are planning to enable priority Queue settings for Q3 by priority 9, Q1 & Q2 have default priority of 0 If an application is triggered in Q3 queue will it get priority over already running applications & can it get maximum cluster resources through preemption even though other queues have applications running in them.
... View more
04-21-2020
07:01 AM
Hi Team, We are using hdp 2.6.4 with capacity scheduler & defaut resource calculator There are 3 parent queues: batch , interactive, default Batch leaf queues are:b1, b2, b3 ( minimum_user_limit% = 20 & ULF= 1 ) Interactive leaf queues are: i1, i2, i3 ( minimum_user_limit% = 20 & ULF= 1 ) In capacity scheduler config we also have these parameters yarn.scheduler.capacity.root.interactive.minimum-user-limit-percent=100 yarn.scheduler.capacity.root.batch.minimum-user-limit-percent=100 yarn.scheduler.capacity.root.batch.user-limit-factor=1 yarn.scheduler.capacity.root.interactive.user-limit-factor=1 specifically to minimum_user_limit , which takes the precedence is it the one set at the leaf level or the one set at the parent level We have seen users using more than the leaf queue's allocated share so wanted to know is it because of the above settings set at the parent leaf level for both minimun_user_limit & ULF. If so can we drop that to 20% & ULF = 0.25? Thanks
... View more