Support Questions
Find answers, ask questions, and share your expertise

Hi, I have a problem with yarn/ranger. I activated the ranger pluggin for yarn but I don't have the log in ranger audit and the policies don't work

Hi, I have a problem with yarn/ranger. I activated the ranger pluggin for yarn but I don't have the log in ranger audit and the policies don't work

New Contributor
 
5 REPLIES 5

Re: Hi, I have a problem with yarn/ranger. I activated the ranger pluggin for yarn but I don't have the log in ranger audit and the policies don't work

Does the yarn plugin download the policies? Do you see any error in ranger logs?

Re: Hi, I have a problem with yarn/ranger. I activated the ranger pluggin for yarn but I don't have the log in ranger audit and the policies don't work

New Contributor

I think it's ok, when I modify the ranger policies for yarn I have this:

09/11/2017 06:34:04 PMProto2_yarnyarn@hostname-Proto2_yarnIP 200 Policies synced to plugin

Re: Hi, I have a problem with yarn/ranger. I activated the ranger pluggin for yarn but I don't have the log in ranger audit and the policies don't work

ok, can you elaborate on what the error is? Do you see any error in resource manager logs?

Re: Hi, I have a problem with yarn/ranger. I activated the ranger pluggin for yarn but I don't have the log in ranger audit and the policies don't work

New Contributor

Hello,

We changed this option: yarn.resourcemanager.scheduler.class : org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler to org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler. When the value of this option is org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler the audit log and the ranger permissions don’t work and when the value is org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler it’s work.

The HFDS component is an isilon and in this doc http://www-01.ibm.com/support/docview.wss?uid=swg27048931 we can see we must used org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler. We must configure this option to FairScheduler because we used an isilon but this option doesn’t work with ranger/yarn. We think it’s a bug.

Someone can help?

Re: Hi, I have a problem with yarn/ranger. I activated the ranger pluggin for yarn but I don't have the log in ranger audit and the policies don't work

New Contributor

I don't see any error in resource manager logs:

$ cat yarn-yarn-resourcemanager-hostname.out

Sep 11, 2017 6:10:36 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register

INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver as a provider class

Sep 11, 2017 6:10:36 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register

INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as a root resource class

Sep 11, 2017 6:10:36 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register

INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class

Sep 11, 2017 6:10:36 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate

INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM'

Sep 11, 2017 6:10:36 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider

INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"

Sep 11, 2017 6:10:36 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider

INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton"

Sep 11, 2017 6:10:37 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider

INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to GuiceManagedComponentProvider with the scope "Singleton"

17/09/11 18:10:56 INFO ipc.Server: Auth successful for appattempt_1505146234776_0001_000001 (auth:SIMPLE)

17/09/11 18:12:44 INFO ipc.Server: Auth successful for appattempt_1505146234776_0002_000001 (auth:SIMPLE)

17/09/11 18:22:37 INFO ipc.Server: Auth successful for appattempt_1505146234776_0003_000001 (auth:SIMPLE)

17/09/11 18:23:16 INFO ipc.Server: Auth successful for appattempt_1505146234776_0004_000001 (auth:SIMPLE)

17/09/11 18:41:31 INFO ipc.Server: Auth successful for appattempt_1505146234776_0005_000001 (auth:SIMPLE)

I followed this procedure to activate the pluggins :

https://github.com/abajwa-hw/security-workshops/blob/master/Setup-ranger-23.md#yarn-audit-exercises-...

and I had the option :

  • ranger.add-yarn-authorization = false

in Custom ranger-yarn-security

When I go to the Yarn Queue Manager I can see in the Access Control and Status: Permissions are managed by Ranger.

Then when I submit a job: run-example --master=yarn --queue test SparkPi 10

I can see my job is completed in Yarn Ressource Manger UI but there is nothing in the ranger audit log.