Reply
Explorer
Posts: 7
Registered: ‎07-27-2015

How to not allow users to access YARN resources? Like launching spark jobs..

I tried to blacklist users by putting them in banned users( YARN Configuration->Banned Users List) list but it didn't work. 

 

How can I do this in a kerberos enabled cluster?

 

By this, I mean the banning the users from accessing HDFS directories, running spark jobs etc.

Cloudera Employee
Posts: 306
Registered: ‎01-16-2014

Re: How to not allow users to access YARN resources? Like launching spark jobs..

For YARN you can setup ACL's on the queues. It is a allow the user/group on the list, not block the user/group on the list.

For HDFS you also have ACL's which is completely separate and works just like any other file system.

 

Wilfred

Explorer
Posts: 7
Registered: ‎07-27-2015

Re: How to not allow users to access YARN resources? Like launching spark jobs..

Hi Wilfred,

 

Can you let me know how I can setup ACL's on the queues for YARN?

 

Thanks.

Cloudera Employee
Posts: 306
Registered: ‎01-16-2014

Re: How to not allow users to access YARN resources? Like launching spark jobs..

It depends on the scheduler you are using. It is here for the Fair Scheduler and here for the Capacity Scheduler. Check for the ACL descriptions on the page.

Both acls have the same format "user,... group,..." (space between user and group list which are separated by commas).

 

Wilfred

 

Explorer
Posts: 7
Registered: ‎07-27-2015

Re: How to not allow users to access YARN resources? Like launching spark jobs..

Thanks for your reply.

 

My intention here is to not allow any users to run spark jobs for example. Since spark runs on Yarn, my assumption was giving only few users access will help secure my cluster and prevent everyone from submitting spark jobs.

 

But even though I set the ACL's , any user is able to submit spark jobs.

 

Any help on how to solve?

 

Thanks, your help is much appreciated.

Cloudera Employee
Posts: 306
Registered: ‎01-16-2014

Re: How to not allow users to access YARN resources? Like launching spark jobs..

Make sure you have an ACL set on the root queue also. ACL's are checked up the tree. If you have access to the parent queue you have access to anything below that. You can not have a * (star) anywhere in the tree.

Also check the admin ACL.

 

Wilfred

Announcements