How to prioritize Ranger permissions to HDFS permissions

[#1]

Hi all,

We are trying to add Ranger in our cluster to secure users access.

After the installation, we configured policies in Ranger with the rights we wanted (see the picture below for the most permissive policy).

Then, we made a CHMOD -R 700 on test folder to verify Ranger's policy effect.

But it seems that Ranger policies are not considered. Typical error we get with the file view :

org.apache.hadoop.security.AccessControlException: Permission denied: user=test, access=READ_EXECUTE, inode="/tmp/RangerTestFolder":admin:hdfs:drwx------

Ranger is well configured and Test Connection is "Connected Successfully" for HDFS. We followed these instructions (+ some forums) :

https://fr.hortonworks.com/blog/best-practices-in-hdfs-authorization-with-apache-ranger/

But the problem is still remaining and we don't know why.

How can we make RANGER permissions more important than HDFS ones ? Because it seems to be the case.

Thanks,

Tom

[#2]

@Tom C

How can we make RANGER permissions more important than HDFS ones ?

When HDFS authorization is configured with Ranger plugin initially it will try to find a ranger hdfs policy matching the access. If policy matching the resource/user or group is found it will be enforced and without checking for hdfs posix level permissions the user will be authorized or denied depending on the policy. Only if no policies are found matching the resource/user or group then it will fallback to hdfs posix level permissions.

Why is not working on the above example?

On your above example I see you created a policy only for resource slash /, if you intended all path you should use slash and wildcard /*

Furthermore the error you show is Permission denied: user=test

Since user is not listed as part of Select User you should make sure test user belongs to hadoop, hdfs or HDFS_Admin groups.

HTH

*** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.

[#3]

Hi @Felix Albani

Thank for your answer.

We already intended the slash and wildcard /* without success unfortunately. We waited a few minutes to let Ranger some time to apply its rules : it didn't change anything.

It really seems that Ranger can't overwrite the HDFS 700 (rwx------) permissions we set before.

We are now working with a specific folder and it's the same problem, but the policy is well configured too :

We are now looking for some parameters in Ranger / HDFS to try and check because Ranger Admin seems to be correctly configured.

Tom C

[#4]

@Tom C Perhaps policies are not sync correctly to the Namenode host. You can quickly check by:

1. In Ranger Admin UI perform a change on any hdfs policy - Add a user for example/

2. Navigate to Ranger Admin UI > Audit > Plugin tab.

3. You should see a new entry at the top - make sure export date is for current time and plugin id should be hdfs (if you dont see this entry showing here after time you made the change this means the hdfs ranger plugin is not downloading policies and probably this explains why is not working as expected)

HTH

[#5]

@Felix Albani

Here you got the screen, no apparent problem :

That's why we think it's about some HDFS or Ranger parameters, all seems good in the Admin console.

[#6]

@Tom C the above service name was <clustername>_hadoop right?

Perhaps is a configuration issue as you mentioned. Since its not working as it should you could also turn log4j.logger.org.apache.ranger.authorization=DEBUG in the hdfs log4j to get more information from logs.

[#7]

@Felix Albani yes it is. All seems good in Ranger Admin view.

You add this in the Custom hdfs-log4j ? The parameter isn't here yet for us in log4j conf.

[#8]

Hi,

Here are some details with Ranger logs (a part of it) :

Usersync log :

INFO PasswordValidator [Thread-64968] - Response [FAILED: [amb_ranger_admin] does not exists.] for user: amb_ranger_admin

INFO PasswordValidator [Thread-64970] - Response [FAILED: [admin] does not exists.] for user: admin

>>> Those are infos to say these users are not UNIX users. That's all the logs we have (hundred times a day). For me, it is not our main problem.

XAPORTAL log :

2018-06-20 09:10:29,691 [http-bio-6080-exec-8] INFO org.apache.ranger.security.handler.RangerAuthenticationProvider (RangerAuthenticationProvider.java:147) - Authentication with SHA-256 failed. Now trying with MD5.

2018-06-20 09:10:29,692 [http-bio-6080-exec-8] INFO org.apache.ranger.security.listener.SpringEventListener (SpringEventListener.java:70) - Login Successful:admin | Ip Address:128.1.227.103 | sessionId=A311983F0813A5EDE4B4F34FDF558628 | Epoch=1529485829692

2018-06-20 09:10:29,698 [http-bio-6080-exec-8] INFO org.apache.ranger.biz.SessionMgr (SessionMgr.java:232) - UserSession Updated to set new Permissions to User: admin

2018-06-20 09:10:29,698 [http-bio-6080-exec-8] INFO org.apache.ranger.biz.SessionMgr (SessionMgr.java:184) - Login Success: loginId=admin, sessionId=51435, sessionId=A311983F0813A5EDE4B4F34FDF558628, requestId=128.1.227.103, epoch=1529485829698

>>> No Warning or Error

CATALINA.OUT log :

RemoteUnixLoginModule: RemoteUnixLoginModule::initialize() has been called with callbackhandler: org.springframework.security.authentication.jaas.AbstractJaasAuthenticationProvider$InternalCallbackHandler@12a420aa

RemoteUnixLoginModule: Created RemoteUnixLoginModule

>>> All good

Access_log is good aswell. We don't see errors, in any Ranger log.

Do you have some tips for HDFS/Ranger configuration check ?

Thanks,

Tom