Ranger Group permissions from LDAP - not working in Hive

[#1]

Hello

We use HDP 2.3 with Ranger 0.5 for managing HIVE permissions. (not for HDFS. Just hive). Ranger (usersync) is configured to use Active Directory and it syncs the users & groups from AD without any issues. In Ranger >>> Settings >>> Users/Groups we see each user and it's corresponding Active Directory groups. When granting authorizations to users for hive access - it works perfect ! However - when using groups (which is way more efficient to manage) it just doesn't work. Permission is always denied. It seems using groups just doesn't work - only users. Again - the sync works and i'm able to see the each users' groups in Ranger - but when i use groups instead of users for hive permissions - it does nothing.

Any ideas why ?

Thanks in advance !

Adi J.

[#2 · Accepted solution]

@Adi Jabkowsky Usually this happens because Hiveserver2 cannot determine which groups the user belongs to. Check your Hiveserver2 log for a message that looks like "No groups for user XXX" where XXX is the user that is being denied access.

If this is the case you'll need to make sure that the OS on the Hiveserver2 node can resolve the groups for that user. Either configure the OS to pull user and group information from Active Directory or set up Hadoop Group Mapping.

[#3]

@Terry Stebbens

Thank you for replying ! It is as if the groups are not even being checked - just the user. When i try to select a table which my group has access to the hiveserver2 log shows:

2016-09-27 18:57:01,331 ERROR [HiveServer2-Handler-Pool: Thread-45]: ql.Driver (SessionState.java:printError(960)) - FAILED: HiveAccessControlException Permission denied: user [adija] does not have [SELECT] privilege on [nifitest/dw_dim_subscriber] org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAccessControlException: Permission denied: user [adija] does not have [SELECT] privilege on [nifitest/dw_dim_subscriber] at org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer.checkPrivileges(RangerHiveAuthorizer.java:352) at org.apache.hadoop.hive.ql.Driver.doAuthorizationV2(Driver.java:779) at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:574) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122) at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1116) at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:110) at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:181) at org.apache.hive.service.cli.operation.Operation.run(Operation.java:257) at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:388) at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:375) at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:274) at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:486) at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313) at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)

2016-09-27 18:57:01,365 WARN [HiveServer2-Handler-Pool: Thread-45]: thrift.ThriftCLIService (ThriftCLIService.java:ExecuteStatement(492)) - Error executing statement: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [adija] does not have [SELECT] privilege on [nifitest/dw_dim_subscriber] at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:315) at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:112) at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:181) at org.apache.hive.service.cli.operation.Operation.run(Operation.java:257) at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:388) at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:375) at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:274) at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:486) at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313) at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAccessControlException: Permission denied: user [adija] does not have [SELECT] privilege on [nifitest/dw_dim_subscriber] at org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer.checkPrivileges(RangerHiveAuthorizer.java:352) at org.apache.hadoop.hive.ql.Driver.doAuthorizationV2(Driver.java:779) at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:574) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122) at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1116) at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:110) ... 15 more

[#4]

@Adi Jabkowsky please check hdfs groups <user-id> resolve to fetch the user and it groups. if not hadoop user group mapping is not correct. you might need to check how at OS level user / group are synced ( e.g.SSSD)

[#5]

@Ramesh Mani thank you for replying. I don't understand why i need to use OS level group mapping for hive. The users do not exist on os and I'm not using the ranger for HDFS permissions. I'm using Ranger Just for granting hive permissions for users from Active Directory LDAP. The users are not created at OS level. They do not exist on the os - they exist only in Active Directory. Also, Ranger is able to see each users' groups from active directory (please see screenshot):

I just want to use those Active Directory groups for granting permissions - instead of users.

To sum up: Ranger is synced with LDAP. Users DO NOT exist on OS. Ranger sees users from LDAP and their corresponding groups. (as in screenshot). I am able to grant LDAP users permissions for hive tables. LDAP users can access hive tables according to the permissions i give them. BUT - if i'm using LDAP groups (which ranger sees) for granting permissions - LDAP users that reside in that group receive permission denied.

[#6]

@Adi Jabkowsky The reason that the users need to exist on the OS (or for you to use Hadoop Group Mapping) is that it is the Hiveserver2 process that gets the username and looks up the groups that user is a member of. It then passes the username and its group membership list to the Ranger Hive plugin (which runs in a thread in the Hiveserver2 process) and this uses the user details to check against a cache of the policies defined for Hive. It is important to understand that the Ranger Hive plugin does not communicate back to the Ranger Admin component during this authorization process. If it did it would be much slower and would make Ranger Admin a single point of failure.

When you synchronize your Active Directory users to Ranger using Ranger UserSync, this is only to allow you to add the users and groups to policies in the Ranger Admin UI, it doesn't then make those users available on the cluster itself. You either need to integrate the OS with Active Directory or use the Hadoop Group Mapping feature to make the users and groups available.

[#7]

Hi

Configuring SSSD OS level did the trick as per your suggestions ! Thank you so much !

[#8]

I have configured both ranger usersync and hadoop group mapping:

-the user X appears to be in group G in the ranger admin menu

-the 'hdfs group X' shows the user X to be in group G

-using 'groups X' the user X is not in group G (SSSD not configured)

Do I still need to configure SSSD to sync group G to make Ranger group policies work?

[#9]

I have the same problem using the latest hdp sandbox.

The GROUP in Ranger is not working, only the users, even if in Ranger it is clearly shown the users/groups relationship, and turn the group/user mapping on for AD authentication/authorization.

The sandbox is configured in convoluted way, I couldn't figure out where the hive server 2 is running, in the virtual machine, or in some docker container? Could someone help?

[#10]

Hi @snowman as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.