Member since
02-29-2016
51
Posts
13
Kudos Received
0
Solutions
08-23-2017
07:26 PM
I am using HDP Sandbox 2.5 vmware image and have kerberized the cluster using the web UI. I have created a user as user1 and this user1 belongs to group1(which also created by me). But the problem is this user and group is not reflecting in Ranger because of which I am not able assign any policy to this user or group. Attaching the logs of usersync process : Any help/clues to resolve this problem is appreciated com.sun.jersey.api.client.UniformInterfaceException: POST http://sandbox.hortonworks.com:6080/service/xusers/users/userinfo returned a response status of 401 Unauthorized
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:686)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:568)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.getUsergroupInfo(PolicyMgrUserGroupBuilder.java:567)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.access$500(PolicyMgrUserGroupBuilder.java:72)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder$2.run(PolicyMgrUserGroupBuilder.java:539)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder$2.run(PolicyMgrUserGroupBuilder.java:535)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.addUserGroupInfo(PolicyMgrUserGroupBuilder.java:535)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.addOrUpdateUser(PolicyMgrUserGroupBuilder.java:340)
at org.apache.ranger.unixusersync.process.UnixUserGroupBuilder.updateSink(UnixUserGroupBuilder.java:140)
at org.apache.ranger.usergroupsync.UserGroupSync.run(UserGroupSync.java:58)
at java.lang.Thread.run(Thread.java:745)
22 Aug 2017 16:45:55 ERROR PolicyMgrUserGroupBuilder [UnixUserSyncThread] - Failed to add User :
com.sun.jersey.api.client.UniformInterfaceException: POST http://sandbox.hortonworks.com:6080/service/users/default returned a response status of 401 Unauthorized
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:686)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:568)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.getMUser(PolicyMgrUserGroupBuilder.java:838)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.access$800(PolicyMgrUserGroupBuilder.java:72)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder$5.run(PolicyMgrUserGroupBuilder.java:811)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder$5.run(PolicyMgrUserGroupBuilder.java:807)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.addMUser(PolicyMgrUserGroupBuilder.java:807)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.addOrUpdateUser(PolicyMgrUserGroupBuilder.java:335)
at org.apache.ranger.unixusersync.process.UnixUserGroupBuilder.updateSink(UnixUserGroupBuilder.java:140)
at org.apache.ranger.usergroupsync.UserGroupSync.run(UserGroupSync.java:58)
at java.lang.Thread.run(Thread.java:745)
22 Aug 2017 16:45:55 ERROR PolicyMgrUserGroupBuilder [UnixUserSyncThread] - Failed to add User Group Info :
com.sun.jersey.api.client.UniformInterfaceException: POST http://sandbox.hortonworks.com:6080/service/xusers/users/userinfo returned a response status of 401 Unauthorized
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:686)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:568)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.getUsergroupInfo(PolicyMgrUserGroupBuilder.java:567)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.access$500(PolicyMgrUserGroupBuilder.java:72)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder$2.run(PolicyMgrUserGroupBuilder.java:539)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder$2.run(PolicyMgrUserGroupBuilder.java:535)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.addUserGroupInfo(PolicyMgrUserGroupBuilder.java:535)
at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.addOrUpdateUser(PolicyMgrUserGroupBuilder.java:340)
at org.apache.ranger.unixusersync.process.UnixUserGroupBuilder.updateSink(UnixUserGroupBuilder.java:140)
at org.apache.ranger.usergroupsync.UserGroupSync.run(UserGroupSync.java:58)
at java.lang.Thread.run(Thread.java:745)
22 Aug 2017 16:45:55 INFO UserGroupSync [UnixUserSyncThread] - End: initial load of user/group from source==>sink
22 Aug 2017 16:45:55 INFO UserGroupSync [UnixUserSyncThread] - Done initializing user/group source and sink
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Ranger
08-23-2017
06:01 PM
@Geoffrey Shelton Okot do you know any solution in which I don't have specify user name. Is there no solution in which policy can be created on group level by specifying only group name ?
... View more
08-22-2017
07:30 AM
No @Geoffrey Shelton Okot I did not change anything.
... View more
08-21-2017
11:03 AM
@Geoffrey Shelton Okot did you get some time to visualize the kms-acls.xml file which I attached in previous comments because the solution which you gave did not work. As I am still not able to set the policies on group level. Please let me know if you have something that we can try out.
... View more
08-18-2017
11:32 AM
If you need it to solve the issue then here's my kms-acls.xml <configuration>
<property>
<name>hadoop.kms.acl.CREATE</name>
<value>*</value>
<description>
ACL for create-key operations.
If the user is not in the GET ACL, the key material is not returned
as part of the response.
</description>
</property>
<property>
<name>hadoop.kms.acl.DELETE</name>
<value>*</value>
<description>
ACL for delete-key operations.
</description>
</property>
<property>
<name>hadoop.kms.acl.ROLLOVER</name>
<value>*</value>
<description>
ACL for rollover-key operations.
If the user is not in the GET ACL, the key material is not returned
as part of the response.
</description>
</property>
<property>
<name>hadoop.kms.acl.GET</name>
<value>*</value>
<description>
ACL for get-key-version and get-current-key operations.
</description>
</property>
<property>
<name>hadoop.kms.acl.GET_KEYS</name>
<value>*</value>
<description>
ACL for get-keys operations.
</description>
</property>
<property>
<name>hadoop.kms.acl.GET_METADATA</name>
<value>*</value>
<description>
ACL for get-key-metadata and get-keys-metadata operations.
</description>
</property>
<property>
<name>hadoop.kms.acl.SET_KEY_MATERIAL</name>
<value>*</value>
<description>
Complementary ACL for CREATE and ROLLOVER operations to allow the client
to provide the key material when creating or rolling a key.
</description>
</property>
<property>
<name>hadoop.kms.acl.GENERATE_EEK</name>
<value>*</value>
<description>
ACL for generateEncryptedKey CryptoExtension operations.
</description>
</property>
<property>
<name>hadoop.kms.acl.DECRYPT_EEK</name>
<value>*</value>
<description>
ACL for decryptEncryptedKey CryptoExtension operations.
</description>
</property>
<property>
<name>default.key.acl.MANAGEMENT</name>
<value>*</value>
<description>
default ACL for MANAGEMENT operations for all key acls that are not
explicitly defined.
</description>
</property>
<property>
<name>default.key.acl.GENERATE_EEK</name>
<value>*</value>
<description>
default ACL for GENERATE_EEK operations for all key acls that are not
explicitly defined.
</description>
</property>
<property>
<name>default.key.acl.DECRYPT_EEK</name>
<value>*</value>
<description>
default ACL for DECRYPT_EEK operations for all key acls that are not
explicitly defined.
</description>
</property>
<property>
<name>default.key.acl.READ</name>
<value>*</value>
<description>
default ACL for READ operations for all key acls that are not
explicitly defined.
</description>
</property>
</configuration>
... View more
08-18-2017
05:04 AM
you can see the "Select Group" coloumn right ? Did you try putting some group name there and tested ? If yes please mention, and if not then please try and then mention.
... View more
08-18-2017
05:02 AM
I checked the kms-acls.xml file and the value for all the properties is * set. Is there any property I need to change to make it work ?
... View more
08-17-2017
06:01 PM
No @Geoffrey Shelton Okot this is not the solution. I want to give the permission on the group level.
... View more
08-17-2017
06:01 PM
I am not really looking for this kind of solution. It works when you give permission at user level but i want it to work on group level. I think I have explained well the question if not let me know what I can do to make it more clear. thanks anyways for your reply.
... View more
08-10-2017
01:54 PM
@Geoffrey Shelton Okot Have you created a keytab for the USER_1 check in I am not using keytab for access. Did USER_1 grab a valide kerberos ticket as the USER_1 run Yes I have created proper ticket using kinit command USER_1 might have been blacklisted by the Ambari property, hadoop.kms.blacklist.DECRYPT_EEK. Thats the most probable reason why you are unable to decrypt being an 'USER_1' user. No I have seen into ranger KMS configs only hdfs user is blacklisted. Did you give read permission to USER_1 to that encryption zone? what do you mean by this ? ============================================================== FYI, As soon as I add USER_1 in the access list. it works just fine. It seems to be not working when group level access is given. attaching some logswhich might help you understand the problem better: <em>#hadoop fs -copyFromLocal test3 /test/1234
copyFromLocal: User:USER_1 not allowed to do 'DECRYPT_EEK' on '1234'
17/08/10 13:48:45 WARN retry.RetryInvocationHandler: Exception while invoking ClientNamenodeProtocolTranslatorPB.complete over null. Not retrying because try once and fail.
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): No lease on /test/1234/test3._COPYING_ (inode 329412): File does not exist. Holder DFSClient_NONMAPREDUCE_-196143005_1 does not have any open files.
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3521)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:3611)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:3578)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:905)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:544)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1552)
at org.apache.hadoop.ipc.Client.call(Client.java:1496)
at org.apache.hadoop.ipc.Client.call(Client.java:1396)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233)
at com.sun.proxy.$Proxy16.complete(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.complete(ClientNamenodeProtocolTranslatorPB.java:501)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:278)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:194)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:176)
at com.sun.proxy.$Proxy17.complete(Unknown Source)
at org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:2361)
at org.apache.hadoop.hdfs.DFSOutputStream.closeImpl(DFSOutputStream.java:2338)
at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2303)
at org.apache.hadoop.hdfs.DFSClient.closeAllFilesBeingWritten(DFSClient.java:947)
at org.apache.hadoop.hdfs.DFSClient.closeOutputStreams(DFSClient.java:979)
at org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:1192)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2852)
at org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2869)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)<br></em>
... View more