Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

CopyFromLocal command fails when run as HDFS user on encryptionZone

Solved Go to solution

CopyFromLocal command fails when run as HDFS user on encryptionZone

Hi,

My folder /user/testUser is encrypted.

When I try to run copyFromLocal command as HDFS user on /user/testUser, i am getting following exception.

Can someone please help me to resolve this.



sudo su --c "hdfs dfs -copyFromLocal test.txt /user/testUser” hdfs

copyFromLocal: User:hdfs not allowed to do 'DECRYPT_EEK' on ’test_key'

17/02/15 00:26:24 ERROR hdfs.DFSClient: Failed to close inode 17777

org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): No lease on /user/testUser/test.txt._COPYING_ (inode 17777): File does not exist. Holder DFSClient_NONMAPREDUCE_1724817926_1 does not have any open files.

at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3659)

at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:3749)

at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:3716)

at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:911)

at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:547)

at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)

at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)

at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)

at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351)

at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:422)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1833)

at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2345)




at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1554)

at org.apache.hadoop.ipc.Client.call(Client.java:1498)

at org.apache.hadoop.ipc.Client.call(Client.java:1398)

at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233)

at com.sun.proxy.$Proxy10.complete(Unknown Source)

at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.complete(ClientNamenodeProtocolTranslatorPB.java:503)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:282)

at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:194)

at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:176)

at com.sun.proxy.$Proxy11.complete(Unknown Source)

at org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:2442)

at org.apache.hadoop.hdfs.DFSOutputStream.closeImpl(DFSOutputStream.java:2419)

at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2384)

at org.apache.hadoop.hdfs.DFSClient.closeAllFilesBeingWritten(DFSClient.java:949)

at org.apache.hadoop.hdfs.DFSClient.closeOutputStreams(DFSClient.java:981)

at org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:1211)

at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2886)

at org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2903)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)
1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: CopyFromLocal command fails when run as HDFS user on encryptionZone

Hello @ssathish,

Can you please share the value of hadoop.kms.blacklist.DECRYPT_EEK from /etc/ranger/kms/conf/dbks-site.xml

Looks like the user 'hdfs' might be blacklisted.

View solution in original post

2 REPLIES 2
Highlighted

Re: CopyFromLocal command fails when run as HDFS user on encryptionZone

Hello @ssathish,

Can you please share the value of hadoop.kms.blacklist.DECRYPT_EEK from /etc/ranger/kms/conf/dbks-site.xml

Looks like the user 'hdfs' might be blacklisted.

View solution in original post

Highlighted

Re: CopyFromLocal command fails when run as HDFS user on encryptionZone

Super Guru

@ssathish

hdfs user is not usually allowed to access encryption keys. This ensures that even Hadoop admin cannot access the encrypted data. Check in your Ranger KMS who is authorized to access keys and use that user. You should not be using HDFS user to access encryption keys.

https://community.hortonworks.com/content/supportkb/49505/how-to-correctly-setup-the-hdfs-encryption...

Don't have an account?
Coming from Hortonworks? Activate your account here