Support Questions
Find answers, ask questions, and share your expertise

Disable -D mapreduce.job.queuename

New Contributor

Hi
Is there a way to disable -D mapreduce.job.queuename option at cluster level or to specific users?
I want users to submit jobs only using placement rules. I am using dynamic resource pools on CM.
Currenlty users are able to submit jobs to any pool using -Dmapreduce.job.queuename option.
Anyother thoughts will be helpful.
Thank You
Harsha

1 REPLY 1

Master Guru

If you go over the rule documentation on http://archive.cloudera.com/cdh5/cdh/5/hadoop/hadoop-yarn/hadoop-yarn-site/FairScheduler.html#Config... you'll notice it speaks of a rule called "specified":

 

"""

  • specified: the app is placed into the queue it requested. If the app requested no queue, i.e. it specified "default", we continue.

"""

 

If your placement rules are allowing a "specified" type rule, then a passed config will be considered. If this rule is absent, we will never look for the passed value.

 

You likely want something like this, rules that look at primary and secondary group named queues if they exist, and otherwise rejects them entirely, but never looks at a specified queue:

 

Screen Shot 2015-12-29 at 11.12.57 AM.png

Or in XML form, the equivalent is roughly:

 

<queuePlacementPolicy>
    <!-- rule name="specified" / Remove this away -->
    <rule name="primaryGroup" create="false" />
    <rule name="nestedUserQueue">
        <rule name="secondaryGroupExistingQueue" create="false" />
    </rule>
    <!-- rule name="default" queue="default"/ Uncomment for added leniency --> 
</queuePlacementPolicy>