Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Capacity Scheduler ACL not working as expected

Capacity Scheduler ACL not working as expected

Hi,

I am stuck in Capacity schedular.

I have created 2 queues and assigned appropriate settings for job submissions.

The issue is all the jobs are going to default queue.

If I want it to go to other queue, I need to set that explicitely when I submit the job shich should not be the case.

 

 

 

These are my queues - 

Queue acls for user : puneet

Queue Operations
=====================
root ADMINISTER_QUEUE
default ADMINISTER_QUEUE,SUBMIT_APPLICATIONS
puneet ADMINISTER_QUEUE,SUBMIT_APPLICATIONS

 

 

 

 

This is my configuration file - 

 

 

<?xml version="1.0"?>
<configuration>
<property>
<name>yarn.scheduler.capacity.root.queues</name>
<value>puneet,default</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.puneet.capacity</name>
<value>50</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.capacity</name>
<value>50</value>
</property>

<property>
<name>yarn.scheduler.capacity.root.puneet.state</name>
<value>RUNNING</value>
</property>

<property>
<name>yarn.scheduler.capacity.root.default.state</name>
<value>RUNNING</value>
</property>

<property>
<name>yarn.scheduler.capacity.root.acl_submit_applications</name>
<value>hadoop,yarn,mapred,hdfs</value>
</property>

<property>
<name>yarn.scheduler.capacity.root.puneet.acl_submit_applications</name>
<value>puneet</value>
</property>

<property>
<name>yarn.scheduler.capacity.root.default.acl_submit_applications</name>
<value>*</value>
</property>

<property>
<name>yarn.scheduler.capacity.root.puneet.acl_administer_queue</name>
<value>puneet</value>
<description>
The ACL of who can administer jobs on the default queue.
</description>
</property>


<property>
<name>yarn.scheduler.capacity.root.default.acl_administer_queue</name>
<value>*</value>
<description>
The ACL of who can administer jobs on the default queue.
</description>
</property>

</configuration>

3 REPLIES 3

Re: Capacity Scheduler ACL not working as expected

Master Guru
Setting permissions over queues will not automatically also map your user's applications on top of that queue. Setting the queue explicitly is required in order to submit to one.

FairScheduler has some automatic queue-placement policy configuration ability, if you're looking for such a feature. See http://archive.cloudera.com/cdh5/cdh/5/hadoop/hadoop-yarn/hadoop-yarn-site/FairScheduler.html#Automa...

The CapacityScheduler does not have such a feature, and requires that you set explicitly the mapreduce.job.queue.name or equivalent configs to reach a queue of desire.

Re: Capacity Scheduler ACL not working as expected

Hi Thanks for the reply.

I was able to figure that but the ACLs are not working.

I had configured that some user xyz should not be able to submit it to queue queue1 but when I set the mapreduce.job.queuename property to queue1, it is getting submitted which should not be the case.

 

I have configured the yarn.acl.enable property to true also for this, but still no effect.

Re: Capacity Scheduler ACL not working as expected

Master Guru
Can you share your ACL configs in full?