Created on 01-23-2019 06:02 AM - edited 09-16-2022 07:05 AM
Hi all. I'll apppreciate for any help with the following issue we encountered with Sentry installation. We have kerberized cluster (with Active Directory implementation).
After succesfully Sentry installation and creating appropriate admin roles users from LDAP supergroup cannot get admin permissions.
Below a short explanation about the case:
Current settings:
For example:
CREATE ROLE admin; GRANT ALL ON SERVER server1 TO ROLE admin WITH GRANT OPTION; GRANT ROLE admin TO GROUP hive; GRANT ROLE admin TO GROUP supergroup;
Also:
CREATE ROLE hive_admin; GRANT ALL ON SERVER server1 TO ROLE hive_admin WITH GRANT OPTION; GRANT ROLE hive_admin TO GROUP hive;
Both users from the LDAP group supergroup can connect to beeline or Hive Metastore by HUE browser without error. Both users can see all databases in Hive and create databases, tables in Hive in any database. These users cannot insert data into table due to the permissions errors:
Application application_1547449479591_0007 failed 2 times due to AM Container for appattempt_1547449479591_0007_000002 exited with exitCode: -1000 For more detailed output, check application tracking page:https://[hostname]:8090/proxy/application_1547449479591_0007/Then, click on links to logs of each attempt. Diagnostics: Application application_1547449479591_0007 initialization failed (exitCode=255) with output: main : command provided 0 main : run as user is hive main : requested yarn user is hive Can't create directory /data1/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied Can't create directory /data2/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied Can't create directory /data3/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied Can't create directory /data4/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied Can't create directory /data5/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied Can't create directory /data6/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied Can't create directory /data7/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied Can't create directory /data8/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied Can't create directory /data9/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied Can't create directory /data10/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied Can't create directory /data11/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied Can't create directory /data12/hadoop/yarn/local/usercache/hive/appcache/application_1547449479591_0007 - Permission denied Did not create any app directories
The users can delete tables.
When one of these users execute admin commands such SHOW ROLES I get the following error:
Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. SentryAccessDeniedException: Access denied to pzeger
The same error when the user connected to beeline:
beeline> !connect "jdbc:hive2://[hostname]:10000/default" Connecting to jdbc:hive2://[hostname]:10000/default Enter username for jdbc:hive2://[hostname]:10000/default: pzeger Enter password for jdbc:hive2://[hostname]:10000/default: ********* Connected to: Apache Hive (version 1.1.0-cdh5.15.0) Driver: Hive JDBC (version 1.1.0-cdh5.15.0) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://[hostname]:1> SHOW ROLES; going to print operations logs printed operations logs going to print operations logs INFO : Compiling command(queryId=hive_20190123124242_ff159215-221e-4732-9d55-a2e917c0917f): SHOW ROLES INFO : Semantic Analysis Completed INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:role, type:string, comment:from deserializer)], properties:null) INFO : Completed compiling command(queryId=hive_20190123124242_ff159215-221e-4732-9d55-a2e917c0917f); Time taken: 0.578 seconds INFO : Executing command(queryId=hive_20190123124242_ff159215-221e-4732-9d55-a2e917c0917f): SHOW ROLES INFO : Starting task [Stage-0:DDL] in serial mode ERROR : Error processing Sentry command: Access denied to pzeger.Please grant admin privilege to pzeger. ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. SentryAccessDeniedException: Access denied to pzeger INFO : Completed executing command(queryId=hive_20190123124242_ff159215-221e-4732-9d55-a2e917c0917f); Time taken: 0.433 seconds printed operations logs Getting log thread is interrupted, since query is done! Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. SentryAccessDeniedException: Access denied to pzeger (state=08S01,code=1) java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. SentryAccessDeniedException: Access denied to pzeger at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:294) at org.apache.hive.beeline.Commands.executeInternal(Commands.java:989) at org.apache.hive.beeline.Commands.execute(Commands.java:1177) at org.apache.hive.beeline.Commands.sql(Commands.java:1091) at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1177) at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1010) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:922) at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:518) at org.apache.hive.beeline.BeeLine.main(BeeLine.java:501) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
I attached Sentry and HiveServer2 logs here.
HiveServer2 log:
12:43:09.883 PM DEBUG SentryTransportFactory [commons-pool-EvictionTimer]: Successfully opened transport org.apache.sentry.core.common.transport.SentryTransportFactory$UgiSaslClientTransport@510e3ab3 to [hostname]/[IP]:8038 12:43:09.883 PM DEBUG SentryTransportPool [commons-pool-EvictionTimer]: [1] created [hostname]:8038 12:44:37.551 PM WARN ThriftCLIService [HiveServer2-Handler-Pool: Thread-78]: Error executing statement: org.apache.hive.service.cli.HiveSQLException: Invalid SessionHandle: SessionHandle [6cbad8fb-8f15-46fa-bc3a-bb6ca217784f] at org.apache.hive.service.cli.session.SessionManager.getSession(SessionManager.java:432) at org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:257) at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:501) at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313) at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:747) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 12:44:37.936 PM DEBUG HiveAuthzBindingHook [HiveServer2-Handler-Pool: Thread-78]: stmtAuthObject.getOperationScope() = CONNECT 12:44:37.936 PM DEBUG HiveAuthzBindingHook [HiveServer2-Handler-Pool: Thread-78]: context.getInputs() = [database:test] 12:44:37.936 PM DEBUG HiveAuthzBindingHook [HiveServer2-Handler-Pool: Thread-78]: context.getOutputs() = [] 12:44:37.937 PM DEBUG SimpleDBPolicyEngine [HiveServer2-Handler-Pool: Thread-78]: Getting permissions for [supergroup, cmreadonly, bigdataanalyst] 12:44:37.938 PM DEBUG SentryTransportPool [HiveServer2-Handler-Pool: Thread-78]: [1] obtained transport [hostname]:8038 12:44:37.938 PM DEBUG SentryTransportPool [HiveServer2-Handler-Pool: Thread-78]: Currently 1 active connections, 9 idle connections 12:44:37.938 PM DEBUG RetryClientInvocationHandler [HiveServer2-Handler-Pool: Thread-78]: Calling listPrivilegesForProvider 12:44:37.993 PM DEBUG SentryTransportPool [HiveServer2-Handler-Pool: Thread-78]: [1] returning [hostname]:8038 12:44:37.993 PM DEBUG SimpleDBPolicyEngine [HiveServer2-Handler-Pool: Thread-78]: result = [server=server1] 12:44:37.994 PM DEBUG HiveAuthzBinding [HiveServer2-Handler-Pool: Thread-78]: Testing mode is false 12:44:37.994 PM WARN HiveAuthzConf [HiveServer2-Handler-Pool: Thread-78]: Using the deprecated config setting hive.sentry.server instead of sentry.hive.server 12:44:37.994 PM WARN HiveAuthzConf [HiveServer2-Handler-Pool: Thread-78]: Using the deprecated config setting hive.sentry.provider instead of sentry.provider 12:44:37.994 PM DEBUG HiveAuthzBinding [HiveServer2-Handler-Pool: Thread-78]: Using authorization provider org.apache.sentry.provider.file.HadoopGroupResourceAuthorizationProvider with resource , policy engine org.apache.sentry.policy.db.SimpleDBPolicyEngine, provider backend SimpleCacheProviderBackend 12:44:38.014 PM DEBUG HiveAuthzBinding [HiveServer2-Handler-Pool: Thread-78]: Going to authorize statement SWITCHDATABASE for subject pzeger 12:44:38.014 PM DEBUG HiveAuthzBinding [HiveServer2-Handler-Pool: Thread-78]: requiredInputPrivileges = {Column=[SELECT, INSERT]} 12:44:38.014 PM DEBUG HiveAuthzBinding [HiveServer2-Handler-Pool: Thread-78]: inputHierarchyList = [[Server [name=server1], Database [name=test], Table [name=*], Column [name=*]]] 12:44:38.014 PM DEBUG HiveAuthzBinding [HiveServer2-Handler-Pool: Thread-78]: requiredOuputPrivileges = {} 12:44:38.014 PM DEBUG HiveAuthzBinding [HiveServer2-Handler-Pool: Thread-78]: outputHierarchyList = [[Server [name=server1], Database [name=test], Table [name=*], Column [name=*]]] 12:44:38.014 PM DEBUG ResourceAuthorizationProvider [HiveServer2-Handler-Pool: Thread-78]: Authorization Request for Subject [name=pzeger] [Server [name=server1], Database [name=test], Table [name=*], Column [name=*]] and [SELECT, INSERT] 12:44:38.015 PM DEBUG SimpleDBPolicyEngine [HiveServer2-Handler-Pool: Thread-78]: Getting permissions for [supergroup, cmreadonly, bigdataanalyst] 12:44:38.016 PM DEBUG SimpleDBPolicyEngine [HiveServer2-Handler-Pool: Thread-78]: result = [server=server1] 12:44:38.019 PM DEBUG ResourceAuthorizationProvider [HiveServer2-Handler-Pool: Thread-78]: ProviderPrivilege server=server1, RequestPrivilege Server=server1->Db=test->Table=*->Column=*->action=select, RoleSet, ActiveRoleSet = [ roles = ALL , Result true 12:44:38.081 PM DEBUG HiveAuthzBinding [HiveServer2-Handler-Pool: Thread-78]: Testing mode is false 12:44:38.081 PM WARN HiveAuthzConf [HiveServer2-Handler-Pool: Thread-78]: Using the deprecated config setting hive.sentry.server instead of sentry.hive.server 12:44:38.081 PM WARN HiveAuthzConf [HiveServer2-Handler-Pool: Thread-78]: Using the deprecated config setting hive.sentry.provider instead of sentry.provider 12:44:38.081 PM DEBUG HiveAuthzBinding [HiveServer2-Handler-Pool: Thread-78]: Using authorization provider org.apache.sentry.provider.file.HadoopGroupResourceAuthorizationProvider with resource , policy engine org.apache.sentry.policy.db.SimpleDBPolicyEngine, provider backend org.apache.sentry.provider.db.SimpleDBProviderBackend 12:44:38.107 PM DEBUG RetryClientInvocationHandler [HiveServer2-Background-Pool: Thread-101]: Calling listRoles 12:44:38.118 PM ERROR RetryClientInvocationHandler [HiveServer2-Background-Pool: Thread-101]: failed to execute listRoles java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.sentry.core.common.transport.RetryClientInvocationHandler.invokeImpl(RetryClientInvocationHandler.java:95) at org.apache.sentry.core.common.transport.SentryClientInvocationHandler.invoke(SentryClientInvocationHandler.java:41) at com.sun.proxy.$Proxy30.listRoles(Unknown Source) at org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask.processRoleDDL(SentryGrantRevokeTask.java:239) at org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask.execute(SentryGrantRevokeTask.java:127) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:99) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2054) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1750) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1503) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1287) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1282) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:236) at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:89) at org.apache.hive.service.cli.operation.SQLOperation$3$1.run(SQLOperation.java:301) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920) at org.apache.hive.service.cli.operation.SQLOperation$3.run(SQLOperation.java:314) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.sentry.provider.db.SentryAccessDeniedException: Access denied to pzeger. Server Stacktrace: org.apache.sentry.provider.db.SentryAccessDeniedException: Access denied to pzeger at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.list_sentry_roles_by_group(SentryPolicyStoreProcessor.java:581) at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1077) at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1062) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:36) at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.apache.sentry.service.thrift.Status.throwIfNotOk(Status.java:113) at org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl.listRolesByGroupName(SentryPolicyServiceClientDefaultImpl.java:161) at org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl.listRoles(SentryPolicyServiceClientDefaultImpl.java:207) ... 28 more 12:44:38.119 PM WARN HiveAuthzConf [HiveServer2-Background-Pool: Thread-101]: Using the deprecated config setting hive.sentry.failure.hooks instead of sentry.hive.failure.hooks 12:44:38.119 PM DEBUG SentryTransportPool [HiveServer2-Background-Pool: Thread-101]: [1] returning [hostname]:8038 12:44:38.119 PM ERROR SentryGrantRevokeTask [HiveServer2-Background-Pool: Thread-101]: Error processing Sentry command: Access denied to pzeger.Please grant admin privilege to pzeger. org.apache.sentry.provider.db.SentryAccessDeniedException: Access denied to pzeger. Server Stacktrace: org.apache.sentry.provider.db.SentryAccessDeniedException: Access denied to pzeger at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.list_sentry_roles_by_group(SentryPolicyStoreProcessor.java:581) at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1077) at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1062) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:36) at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.apache.sentry.service.thrift.Status.throwIfNotOk(Status.java:113) at org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl.listRolesByGroupName(SentryPolicyServiceClientDefaultImpl.java:161) at org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl.listRoles(SentryPolicyServiceClientDefaultImpl.java:207) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.sentry.core.common.transport.RetryClientInvocationHandler.invokeImpl(RetryClientInvocationHandler.java:95) at org.apache.sentry.core.common.transport.SentryClientInvocationHandler.invoke(SentryClientInvocationHandler.java:41) at com.sun.proxy.$Proxy30.listRoles(Unknown Source) at org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask.processRoleDDL(SentryGrantRevokeTask.java:239) at org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask.execute(SentryGrantRevokeTask.java:127) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:99) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2054) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1750) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1503) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1287) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1282) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:236) at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:89) at org.apache.hive.service.cli.operation.SQLOperation$3$1.run(SQLOperation.java:301) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920) at org.apache.hive.service.cli.operation.SQLOperation$3.run(SQLOperation.java:314) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 12:44:38.119 PM ERROR Task [HiveServer2-Background-Pool: Thread-101]: Error processing Sentry command: Access denied to pzeger.Please grant admin privilege to pzeger. [HiveServer2-Background-Pool: Thread-101]: </PERFLOG method=FailureHook.com.cloudera.navigator.audit.hive.FailedHiveExecHookContext start=1548240278119 end=1548240278122 duration=3 from=org.apache.hadoop.hive.ql.Driver> 12:44:38.122 PM ERROR Driver [HiveServer2-Background-Pool: Thread-101]: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. SentryAccessDeniedException: Access denied to pzeger [HiveServer2-Background-Pool: Thread-101]: </PERFLOG method=releaseLocks start=1548240278122 end=1548240278122 duration=0 from=org.apache.hadoop.hive.ql.Driver> 12:44:38.128 PM ERROR Operation [HiveServer2-Background-Pool: Thread-101]: Error running hive query: org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. SentryAccessDeniedException: Access denied to pzeger at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:400) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:238) at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:89) at org.apache.hive.service.cli.operation.SQLOperation$3$1.run(SQLOperation.java:301) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920) at org.apache.hive.service.cli.operation.SQLOperation$3.run(SQLOperation.java:314) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.Exception: SentryAccessDeniedException: Access denied to pzeger at org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask.execute(SentryGrantRevokeTask.java:161) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:214) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:99) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2054) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1750) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1503) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1287) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1282) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:236) ... 11 more Caused by: org.apache.sentry.provider.db.SentryAccessDeniedException: Access denied to pzeger. Server Stacktrace: org.apache.sentry.provider.db.SentryAccessDeniedException: Access denied to pzeger at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.list_sentry_roles_by_group(SentryPolicyStoreProcessor.java:581) at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1077) at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1062) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:36) at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.apache.sentry.service.thrift.Status.throwIfNotOk(Status.java:113) at org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl.listRolesByGroupName(SentryPolicyServiceClientDefaultImpl.java:161) at org.apache.sentry.provider.db.service.thrift.SentryPolicyServiceClientDefaultImpl.listRoles(SentryPolicyServiceClientDefaultImpl.java:207) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.sentry.core.common.transport.RetryClientInvocationHandler.invokeImpl(RetryClientInvocationHandler.java:95) at org.apache.sentry.core.common.transport.SentryClientInvocationHandler.invoke(SentryClientInvocationHandler.java:41) at com.sun.proxy.$Proxy30.listRoles(Unknown Source) at org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask.processRoleDDL(SentryGrantRevokeTask.java:239) at org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask.execute(SentryGrantRevokeTask.java:127) ... 19 more
Sentry log:
12:44:37.985 PM INFO Query Reading in results for query "SELECT FROM org.apache.sentry.provider.db.service.model.MSentryPrivilege WHERE (roles.contains(role) && this.serverName == :serverName && (role.roleName == :var0)) VARIABLES org.apache.sentry.provider.db.service.model.MSentryRole role" since the connection used is closing 12:44:38.116 PM WARN ShellBasedUnixGroupsMapping unable to return groups for user pzeger PartialGroupNameException The user name 'pzeger' is not found. id: 'pzeger': no such user id: 'pzeger': no such user at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.resolvePartialGroupNames(ShellBasedUnixGroupsMapping.java:212) at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:133) at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:72) at org.apache.hadoop.security.Groups$GroupCacheLoader.fetchGroupList(Groups.java:371) at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:311) at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:269) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) at com.google.common.cache.LocalCache.get(LocalCache.java:3965) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) at org.apache.hadoop.security.Groups.getGroups(Groups.java:227) at org.apache.sentry.provider.common.HadoopGroupMappingService.getGroups(HadoopGroupMappingService.java:60) at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.getGroupsFromUserName(SentryPolicyStoreProcessor.java:737) at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.getRequestorGroups(SentryPolicyStoreProcessor.java:704) at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.list_sentry_roles_by_group(SentryPolicyStoreProcessor.java:572) at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1077) at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1062) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:36) at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 12:44:38.117 PM WARN HadoopGroupMappingService Unable to obtain groups for pzeger java.io.IOException: No groups found for user pzeger at org.apache.hadoop.security.Groups.noGroupsForUser(Groups.java:199) at org.apache.hadoop.security.Groups.access$400(Groups.java:74) at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:319) at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:269) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) at com.google.common.cache.LocalCache.get(LocalCache.java:3965) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) at org.apache.hadoop.security.Groups.getGroups(Groups.java:227) at org.apache.sentry.provider.common.HadoopGroupMappingService.getGroups(HadoopGroupMappingService.java:60) at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.getGroupsFromUserName(SentryPolicyStoreProcessor.java:737) at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.getRequestorGroups(SentryPolicyStoreProcessor.java:704) at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.list_sentry_roles_by_group(SentryPolicyStoreProcessor.java:572) at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1077) at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1062) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:36) at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 12:44:38.117 PM ERROR SentryPolicyStoreProcessor Access denied to pzeger org.apache.sentry.provider.db.SentryAccessDeniedException: Access denied to pzeger at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.list_sentry_roles_by_group(SentryPolicyStoreProcessor.java:581) at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1077) at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1062) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:36) at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 12:46:07.424 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31620 and being sent to HDFS 12:46:07.925 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31621 and being sent to HDFS 12:46:07.928 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31621 and being sent to HDFS 12:46:08.429 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PermImageRetriever) Newer delta updates are found up to sequence number 10559 and being sent to HDFS 12:46:08.431 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PermImageRetriever) Newer delta updates are found up to sequence number 10559 and being sent to HDFS 12:46:08.432 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31624 and being sent to HDFS 12:46:08.934 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PermImageRetriever) Newer delta updates are found up to sequence number 10560 and being sent to HDFS 12:46:08.935 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PermImageRetriever) Newer delta updates are found up to sequence number 10560 and being sent to HDFS 12:46:08.936 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31625 and being sent to HDFS 12:46:09.439 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31625 and being sent to HDFS 12:46:12.449 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31626 and being sent to HDFS 12:46:12.950 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31627 and being sent to HDFS 12:46:13.453 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PermImageRetriever) Newer delta updates are found up to sequence number 10561 and being sent to HDFS 12:46:13.453 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PermImageRetriever) Newer delta updates are found up to sequence number 10561 and being sent to HDFS 12:46:13.456 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31630 and being sent to HDFS 12:46:13.956 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PermImageRetriever) Newer delta updates are found up to sequence number 10562 and being sent to HDFS 12:46:13.959 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PermImageRetriever) Newer delta updates are found up to sequence number 10562 and being sent to HDFS 12:46:13.959 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31631 and being sent to HDFS 12:46:14.462 PM INFO DBUpdateForwarder (org.apache.sentry.hdfs.PathImageRetriever) Newer delta updates are found up to sequence number 31631 and being sent to HDFS
As I see despite the implementation of LDAP groups mapping in Hadoop when Sentry uses the same group mechanism configured in HDFS service, the Sentry service warns about ShellBasedUnixGroupsMapping instead of LdapGroupsMapping. I also see in the log that Hive succesfully recognize LDAP groups such supergroup, cmreadonly and etc.
Created 11-29-2020 09:46 PM
Hi @PavelZeger , Did you find any solution for this , I am also trying to enable sentry with LDAP but facing issues.