Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Sentry don't synchronize with LDAP

avatar
Explorer

Hi all. I'll apppreciate for any help with the following issue we encountered with Sentry installation. We have kerberized cluster (with Active Directory implementation).

After succesfully Sentry installation and creating appropriate admin roles users from LDAP supergroup cannot get admin permissions.

Below a short explanation about the case:

 

Current settings:

  • security.group.mapping: org.apache.hadoop.security.LdapGroupsMapping.
  • Hive Sentry User to Group Mapping Class: org.apache.sentry.provider.file.HadoopGroupResourceAuthorizationProvider
  • Sentry Admin Groups includes supergroup.
  • LDAP supergroup includes: pzeger (user for checks).
  • Hive configured to authenticate through LDAP.
  • HUE configurations with LDAP allow to synchronize the supergroup to HUE service without error.
  • Sentry was configured according to the Cloudera documentation.
  • Also HUE includes the following configuration in order to prevent HUE connection to Hive by LDAP and not by Kerberos (anyway without this configuration I'm getting the same error): hive.server2.authentication > kerberos
  • I'm able to connect to beeline with the user hive without any error and also I can create any role and associate it with any group.

For example:

 

CREATE ROLE admin;
GRANT ALL ON SERVER server1 TO ROLE admin WITH GRANT OPTION;
GRANT ROLE admin TO GROUP hive;
GRANT ROLE admin TO GROUP supergroup;

 

Also:

 

CREATE ROLE hive_admin;
GRANT ALL ON SERVER server1 TO ROLE hive_admin WITH GRANT OPTION;
GRANT ROLE hive_admin TO GROUP hive;

Both users from the LDAP group supergroup can connect to beeline or Hive Metastore by HUE browser without error. Both users can see all databases in Hive and create databases, tables in Hive in any database. These users cannot insert data into table due to the permissions errors:

  

Application application_1547449479591_0007 failed 2 times due to AM Container for appattempt_1547449479591_0007_000002 exited with  exitCode: -1000
For more detailed output, check application tracking page:https://[hostname]:8090/proxy/application_1547449479591_0007/Then, click on links to logs of each attempt.
Diagnostics: Application application_1547449479591_0007 initialization failed (exitCode=255) with output: main : command provided 0
main : run as user is hive
main : requested yarn user is hive
Can't create directory /data1/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied
Can't create directory /data2/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied
Can't create directory /data3/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied
Can't create directory /data4/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied
Can't create directory /data5/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied
Can't create directory /data6/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied
Can't create directory /data7/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied
Can't create directory /data8/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied
Can't create directory /data9/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied
Can't create directory /data10/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied
Can't create directory /data11/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied
Can't create directory /data12/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied
Did not create any app directories

The users can delete tables.

 

When one of these users execute admin commands such SHOW ROLES I get the following error:

Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. SentryAccessDeniedException: Access denied to pzeger

 

The same error when the user connected to beeline:

 

beeline> !connect "jdbc:hive2://[hostname]:10000/default"
Connecting to jdbc:hive2://[hostname]:10000/default
Enter username for jdbc:hive2://[hostname]:10000/default: pzeger
Enter password for jdbc:hive2://[hostname]:10000/default: *********
Connected to: Apache Hive (version 1.1.0-cdh5.15.0)
Driver: Hive JDBC (version 1.1.0-cdh5.15.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://[hostname]:1> SHOW ROLES;
going to print operations logs
printed operations logs
going to print operations logs
INFO  : Compiling command(queryId=hive_20190123124242_ff159215-221e-4732-9d55-a2e917c0917f): SHOW ROLES
INFO  : Semantic Analysis Completed
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:role, type:string, comment:from deserializer)], properties:null)
INFO  : Completed compiling command(queryId=hive_20190123124242_ff159215-221e-4732-9d55-a2e917c0917f); Time taken: 0.578 seconds
INFO  : Executing command(queryId=hive_20190123124242_ff159215-221e-4732-9d55-a2e917c0917f): SHOW ROLES
INFO  : Starting task [Stage-0:DDL] in serial mode
ERROR : Error processing Sentry command: Access denied to pzeger.Please grant admin privilege to pzeger.
ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. SentryAccessDeniedException: Access denied to pzeger
INFO  : Completed executing command(queryId=hive_20190123124242_ff159215-221e-4732-9d55-a2e917c0917f); Time taken: 0.433 seconds
printed operations logs
Getting log thread is interrupted, since query is done!
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. SentryAccessDeniedException: Access denied to pzeger (state=08S01,code=1)
java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. SentryAccessDeniedException: Access denied to pzeger
        at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:294)
        at org.apache.hive.beeline.Commands.executeInternal(Commands.java:989)
        at org.apache.hive.beeline.Commands.execute(Commands.java:1177)
        at org.apache.hive.beeline.Commands.sql(Commands.java:1091)
        at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1177)
        at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1010)
        at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:922)
        at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:518)
        at org.apache.hive.beeline.BeeLine.main(BeeLine.java:501)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

I attached Sentry and HiveServer2 logs here.

HiveServer2 log:

 

12:43:09.883 PM               DEBUG SentryTransportFactory               
[commons-pool-EvictionTimer]: Successfully opened transport org.apache.sentry.core.common.transport.SentryTransportFactory$UgiSaslClientTransport@510e3ab3 to [hostname]/[IP]:8038
12:43:09.883 PM               DEBUG SentryTransportPool     
[commons-pool-EvictionTimer]: [1] created [hostname]:8038
12:44:37.551 PM               WARN   ThriftCLIService
[HiveServer2-Handler-Pool: Thread-78]: Error executing statement:
org.apache.hive.service.cli.HiveSQLException: Invalid SessionHandle: SessionHandle [6cbad8fb-8f15-46fa-bc3a-bb6ca217784f]
                at org.apache.hive.service.cli.session.SessionManager.getSession(SessionManager.java:432)
                at org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:257)
                at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:501)
                at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
                at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
                at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
                at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
                at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:747)
                at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at java.lang.Thread.run(Thread.java:748)
12:44:37.936 PM               DEBUG HiveAuthzBindingHook
[HiveServer2-Handler-Pool: Thread-78]: stmtAuthObject.getOperationScope() = CONNECT
12:44:37.936 PM               DEBUG HiveAuthzBindingHook
[HiveServer2-Handler-Pool: Thread-78]: context.getInputs() = [database:test]
12:44:37.936 PM               DEBUG HiveAuthzBindingHook
[HiveServer2-Handler-Pool: Thread-78]: context.getOutputs() = []
12:44:37.937 PM               DEBUG SimpleDBPolicyEngine   
[HiveServer2-Handler-Pool: Thread-78]: Getting permissions for [supergroup, cmreadonly, bigdataanalyst]
12:44:37.938 PM               DEBUG SentryTransportPool     
[HiveServer2-Handler-Pool: Thread-78]: [1] obtained transport [hostname]:8038
12:44:37.938 PM               DEBUG SentryTransportPool     
[HiveServer2-Handler-Pool: Thread-78]: Currently 1 active connections, 9 idle connections
12:44:37.938 PM               DEBUG RetryClientInvocationHandler   
[HiveServer2-Handler-Pool: Thread-78]: Calling listPrivilegesForProvider
12:44:37.993 PM               DEBUG SentryTransportPool     
[HiveServer2-Handler-Pool: Thread-78]: [1] returning [hostname]:8038
12:44:37.993 PM               DEBUG SimpleDBPolicyEngine   
[HiveServer2-Handler-Pool: Thread-78]: result = [server=server1]
12:44:37.994 PM               DEBUG HiveAuthzBinding           
[HiveServer2-Handler-Pool: Thread-78]: Testing mode is false
12:44:37.994 PM               WARN   HiveAuthzConf 
[HiveServer2-Handler-Pool: Thread-78]: Using the deprecated config setting hive.sentry.server instead of sentry.hive.server
12:44:37.994 PM               WARN   HiveAuthzConf 
[HiveServer2-Handler-Pool: Thread-78]: Using the deprecated config setting hive.sentry.provider instead of sentry.provider
12:44:37.994 PM               DEBUG HiveAuthzBinding           
[HiveServer2-Handler-Pool: Thread-78]: Using authorization provider org.apache.sentry.provider.file.HadoopGroupResourceAuthorizationProvider with resource , policy engine org.apache.sentry.policy.db.SimpleDBPolicyEngine, provider backend SimpleCacheProviderBackend
12:44:38.014 PM               DEBUG HiveAuthzBinding           
[HiveServer2-Handler-Pool: Thread-78]: Going to authorize statement SWITCHDATABASE for subject pzeger
12:44:38.014 PM               DEBUG HiveAuthzBinding           
[HiveServer2-Handler-Pool: Thread-78]: requiredInputPrivileges = {Column=[SELECT, INSERT]}
12:44:38.014 PM               DEBUG HiveAuthzBinding           
[HiveServer2-Handler-Pool: Thread-78]: inputHierarchyList = [[Server [name=server1], Database [name=test], Table [name=*], Column [name=*]]]
12:44:38.014 PM               DEBUG HiveAuthzBinding           
[HiveServer2-Handler-Pool: Thread-78]: requiredOuputPrivileges = {}
12:44:38.014 PM               DEBUG HiveAuthzBinding           
[HiveServer2-Handler-Pool: Thread-78]: outputHierarchyList = [[Server [name=server1], Database [name=test], Table [name=*], Column [name=*]]]
12:44:38.014 PM               DEBUG ResourceAuthorizationProvider
[HiveServer2-Handler-Pool: Thread-78]: Authorization Request for Subject [name=pzeger] [Server [name=server1], Database [name=test], Table [name=*], Column [name=*]] and [SELECT, INSERT]
12:44:38.015 PM               DEBUG SimpleDBPolicyEngine   
[HiveServer2-Handler-Pool: Thread-78]: Getting permissions for [supergroup, cmreadonly, bigdataanalyst]
12:44:38.016 PM               DEBUG SimpleDBPolicyEngine   
[HiveServer2-Handler-Pool: Thread-78]: result = [server=server1]
12:44:38.019 PM               DEBUG ResourceAuthorizationProvider
[HiveServer2-Handler-Pool: Thread-78]: ProviderPrivilege server=server1, RequestPrivilege Server=server1->Db=test->Table=*->Column=*->action=select, RoleSet, ActiveRoleSet = [ roles = ALL , Result true
12:44:38.081 PM               DEBUG HiveAuthzBinding           
[HiveServer2-Handler-Pool: Thread-78]: Testing mode is false
12:44:38.081 PM               WARN   HiveAuthzConf 
[HiveServer2-Handler-Pool: Thread-78]: Using the deprecated config setting hive.sentry.server instead of sentry.hive.server
12:44:38.081 PM               WARN   HiveAuthzConf 
[HiveServer2-Handler-Pool: Thread-78]: Using the deprecated config setting hive.sentry.provider instead of sentry.provider
12:44:38.081 PM               DEBUG HiveAuthzBinding           
[HiveServer2-Handler-Pool: Thread-78]: Using authorization provider org.apache.sentry.provider.file.HadoopGroupResourceAuthorizationProvider with resource , policy engine org.apache.sentry.policy.db.SimpleDBPolicyEngine, provider backend org.apache.sentry.provider.db.SimpleDBProviderBackend
12:44:38.107 PM               DEBUG RetryClientInvocationHandler   
[HiveServer2-Background-Pool: Thread-101]: Calling listRoles
12:44:38.118 PM               ERROR  RetryClientInvocationHandler   
[HiveServer2-Background-Pool: Thread-101]: failed to execute listRoles
java.lang.reflect.InvocationTargetException
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.lang.reflect.Method.invoke(Method.java:498)
                at org.apache.sentry.core.common.transport.RetryClientInvocationHandler.invokeImpl(RetryClientInvocationHandler.java:95)
                at org.apache.sentry.core.common.transport.SentryClientInvocationHandler.invoke(SentryClientInvocationHandler.java:41)
                at com.sun.proxy.$Proxy30.listRoles(Unknown Source)
                at org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask.processRoleDDL(SentryGrantRevokeTask.java:239)
                at org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask.execute(SentryGrantRevokeTask.java:127)
                at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214)
                at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:99)
                at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2054)
                at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1750)
                at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1503)
                at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1287)
                at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1282)
                at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:236)
                at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:89)
                at org.apache.hive.service.cli.operation.SQLOperation$3$1.run(SQLOperation.java:301)
                at java.security.AccessController.doPrivileged(Native Method)
                at javax.security.auth.Subject.doAs(Subject.java:422)
                at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
                at org.apache.hive.service.cli.operation.SQLOperation$3.run(SQLOperation.java:314)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
                at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.sentry.provider.db.SentryAccessDeniedException: Access denied to pzeger. Server Stacktrace: org.apache.sentry.provider.db.SentryAccessDeniedException: Access denied to pzeger
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.list_sentry_roles_by_group(SentryPolicyStoreProcessor.java:581)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1077)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1062)
                at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
                at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
                at org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:36)
                at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
                at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at java.lang.Thread.run(Thread.java:748)
                at org.apache.sentry.service.thrift.Status.throwIfNotOk(Status.java:113)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl.listRolesByGroupName(SentryPolicyServiceClientDefaultImpl.java:161)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl.listRoles(SentryPolicyServiceClientDefaultImpl.java:207)
                ... 28 more
12:44:38.119 PM               WARN   HiveAuthzConf 
[HiveServer2-Background-Pool: Thread-101]: Using the deprecated config setting hive.sentry.failure.hooks instead of sentry.hive.failure.hooks
12:44:38.119 PM               DEBUG SentryTransportPool     
[HiveServer2-Background-Pool: Thread-101]: [1] returning [hostname]:8038
12:44:38.119 PM               ERROR  SentryGrantRevokeTask              
[HiveServer2-Background-Pool: Thread-101]: Error processing Sentry command: Access denied to pzeger.Please grant admin privilege to pzeger.
org.apache.sentry.provider.db.SentryAccessDeniedException: Access denied to pzeger. Server Stacktrace: org.apache.sentry.provider.db.SentryAccessDeniedException: Access denied to pzeger
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.list_sentry_roles_by_group(SentryPolicyStoreProcessor.java:581)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1077)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1062)
                at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
                at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
                at org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:36)
                at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
                at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at java.lang.Thread.run(Thread.java:748)
                at org.apache.sentry.service.thrift.Status.throwIfNotOk(Status.java:113)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl.listRolesByGroupName(SentryPolicyServiceClientDefaultImpl.java:161)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl.listRoles(SentryPolicyServiceClientDefaultImpl.java:207)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.lang.reflect.Method.invoke(Method.java:498)
                at org.apache.sentry.core.common.transport.RetryClientInvocationHandler.invokeImpl(RetryClientInvocationHandler.java:95)
                at org.apache.sentry.core.common.transport.SentryClientInvocationHandler.invoke(SentryClientInvocationHandler.java:41)
                at com.sun.proxy.$Proxy30.listRoles(Unknown Source)
                at org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask.processRoleDDL(SentryGrantRevokeTask.java:239)
                at org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask.execute(SentryGrantRevokeTask.java:127)
                at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214)
                at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:99)
                at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2054)
                at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1750)
                at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1503)
                at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1287)
                at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1282)
                at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:236)
                at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:89)
                at org.apache.hive.service.cli.operation.SQLOperation$3$1.run(SQLOperation.java:301)
                at java.security.AccessController.doPrivileged(Native Method)
                at javax.security.auth.Subject.doAs(Subject.java:422)
                at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
                at org.apache.hive.service.cli.operation.SQLOperation$3.run(SQLOperation.java:314)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
                at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at java.lang.Thread.run(Thread.java:748)
12:44:38.119 PM               ERROR  Task      
[HiveServer2-Background-Pool: Thread-101]: Error processing Sentry command: Access denied to pzeger.Please grant admin privilege to pzeger.        
[HiveServer2-Background-Pool: Thread-101]: </PERFLOG method=FailureHook.com.cloudera.navigator.audit.hive.FailedHiveExecHookContext start=1548240278119 end=1548240278122 duration=3 from=org.apache.hadoop.hive.ql.Driver>
12:44:38.122 PM               ERROR  Driver   
[HiveServer2-Background-Pool: Thread-101]: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. SentryAccessDeniedException: Access denied to pzeger
[HiveServer2-Background-Pool: Thread-101]: </PERFLOG method=releaseLocks start=1548240278122 end=1548240278122 duration=0 from=org.apache.hadoop.hive.ql.Driver>
12:44:38.128 PM               ERROR  Operation          
[HiveServer2-Background-Pool: Thread-101]: Error running hive query:
org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. SentryAccessDeniedException: Access denied to pzeger
                at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:400)
                at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:238)
                at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:89)
                at org.apache.hive.service.cli.operation.SQLOperation$3$1.run(SQLOperation.java:301)
                at java.security.AccessController.doPrivileged(Native Method)
                at javax.security.auth.Subject.doAs(Subject.java:422)
                at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
                at org.apache.hive.service.cli.operation.SQLOperation$3.run(SQLOperation.java:314)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
                at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.Exception: SentryAccessDeniedException: Access denied to pzeger
                at org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask.execute(SentryGrantRevokeTask.java:161)
                at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214)
                at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:99)
                at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2054)
                at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1750)
                at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1503)
                at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1287)
                at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1282)
                at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:236)
                ... 11 more
Caused by: org.apache.sentry.provider.db.SentryAccessDeniedException: Access denied to pzeger. Server Stacktrace: org.apache.sentry.provider.db.SentryAccessDeniedException: Access denied to pzeger
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.list_sentry_roles_by_group(SentryPolicyStoreProcessor.java:581)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1077)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1062)
                at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
                at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
                at org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:36)
                at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
                at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at java.lang.Thread.run(Thread.java:748)
                at org.apache.sentry.service.thrift.Status.throwIfNotOk(Status.java:113)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl.listRolesByGroupName(SentryPolicyServiceClientDefaultImpl.java:161)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl.listRoles(SentryPolicyServiceClientDefaultImpl.java:207)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.lang.reflect.Method.invoke(Method.java:498)
                at org.apache.sentry.core.common.transport.RetryClientInvocationHandler.invokeImpl(RetryClientInvocationHandler.java:95)
                at org.apache.sentry.core.common.transport.SentryClientInvocationHandler.invoke(SentryClientInvocationHandler.java:41)
                at com.sun.proxy.$Proxy30.listRoles(Unknown Source)
                at org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask.processRoleDDL(SentryGrantRevokeTask.java:239)
                at org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask.execute(SentryGrantRevokeTask.java:127)
                ... 19 more

Sentry log:

 

12:44:37.985 PM               INFO      Query   
Reading in results for query "SELECT FROM org.apache.sentry.provider.db.service.model.MSentryPrivilege WHERE (roles.contains(role) && this.serverName == :serverName && (role.roleName == :var0)) VARIABLES org.apache.sentry.provider.db.service.model.MSentryRole role" since the connection used is closing
12:44:38.116 PM               WARN   ShellBasedUnixGroupsMapping              
unable to return groups for user pzeger
PartialGroupNameException The user name 'pzeger' is not found. id: 'pzeger': no such user
id: 'pzeger': no such user
                at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.resolvePartialGroupNames(ShellBasedUnixGroupsMapping.java:212)
                at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:133)
                at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:72)
                at org.apache.hadoop.security.Groups$GroupCacheLoader.fetchGroupList(Groups.java:371)
                at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:311)
                at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:269)
                at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
                at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
                at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
                at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
                at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
                at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969)
                at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829)
                at org.apache.hadoop.security.Groups.getGroups(Groups.java:227)
                at org.apache.sentry.provider.common.HadoopGroupMappingService.getGroups(HadoopGroupMappingService.java:60)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.getGroupsFromUserName(SentryPolicyStoreProcessor.java:737)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.getRequestorGroups(SentryPolicyStoreProcessor.java:704)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.list_sentry_roles_by_group(SentryPolicyStoreProcessor.java:572)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1077)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1062)
                at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
                at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
                at org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:36)
                at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
                at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at java.lang.Thread.run(Thread.java:748)
12:44:38.117 PM               WARN   HadoopGroupMappingService 
Unable to obtain groups for pzeger
java.io.IOException: No groups found for user pzeger
                at org.apache.hadoop.security.Groups.noGroupsForUser(Groups.java:199)
                at org.apache.hadoop.security.Groups.access$400(Groups.java:74)
                at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:319)
                at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:269)
                at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
                at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
                at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
                at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
                at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
                at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969)
                at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829)
                at org.apache.hadoop.security.Groups.getGroups(Groups.java:227)
                at org.apache.sentry.provider.common.HadoopGroupMappingService.getGroups(HadoopGroupMappingService.java:60)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.getGroupsFromUserName(SentryPolicyStoreProcessor.java:737)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.getRequestorGroups(SentryPolicyStoreProcessor.java:704)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.list_sentry_roles_by_group(SentryPolicyStoreProcessor.java:572)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1077)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1062)
                at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
                at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
                at org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:36)
                at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
                at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at java.lang.Thread.run(Thread.java:748)
12:44:38.117 PM               ERROR  SentryPolicyStoreProcessor        
Access denied to pzeger
org.apache.sentry.provider.db.SentryAccessDeniedException: Access denied to pzeger
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.list_sentry_roles_by_group(SentryPolicyStoreProcessor.java:581)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1077)
                at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1062)
                at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
                at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
                at org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:36)
                at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
                at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at java.lang.Thread.run(Thread.java:748)
12:46:07.424 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31620 and being sent to HDFS
12:46:07.925 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31621 and being sent to HDFS
12:46:07.928 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31621 and being sent to HDFS
12:46:08.429 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PermImageRetriever) Newer delta updates are found up to sequence number 10559 and being sent to HDFS
12:46:08.431 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PermImageRetriever) Newer delta updates are found up to sequence number 10559 and being sent to HDFS
12:46:08.432 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31624 and being sent to HDFS
12:46:08.934 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PermImageRetriever) Newer delta updates are found up to sequence number 10560 and being sent to HDFS
12:46:08.935 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PermImageRetriever) Newer delta updates are found up to sequence number 10560 and being sent to HDFS
12:46:08.936 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31625 and being sent to HDFS
12:46:09.439 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31625 and being sent to HDFS
12:46:12.449 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31626 and being sent to HDFS
12:46:12.950 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31627 and being sent to HDFS
12:46:13.453 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PermImageRetriever) Newer delta updates are found up to sequence number 10561 and being sent to HDFS
12:46:13.453 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PermImageRetriever) Newer delta updates are found up to sequence number 10561 and being sent to HDFS
12:46:13.456 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31630 and being sent to HDFS
12:46:13.956 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PermImageRetriever) Newer delta updates are found up to sequence number 10562 and being sent to HDFS
12:46:13.959 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PermImageRetriever) Newer delta updates are found up to sequence number 10562 and being sent to HDFS
12:46:13.959 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31631 and being sent to HDFS
12:46:14.462 PM               INFO      DBUpdateForwarder     
(org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31631 and being sent to HDFS

 

As I see despite the implementation of LDAP groups mapping in Hadoop when Sentry uses the same group mechanism configured in HDFS service, the Sentry service warns about ShellBasedUnixGroupsMapping instead of LdapGroupsMapping. I also see in the log that Hive succesfully recognize LDAP groups such supergroup, cmreadonly and etc.

 

1 REPLY 1

avatar
New Contributor

Hi @PavelZeger , Did you find any solution for this , I am also trying to enable sentry with LDAP but facing issues.