Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDFS Encryption - error in loading data into encryption zone (org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException)

HDFS Encryption - error in loading data into encryption zone (org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException)

Expert Contributor

Hi - i've non-kerberized HDP 2.4 cluster, and i'm trying to evalaute/implement HDFS encryption.

I've created a encryption key & encryption zone.

When i try to add a file into encryption_zone, it goves error shown below

Reference ->

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.3/bk_Security_Guide/content/copy-to-from-encr...

https://github.com/abajwa-hw/security-workshops/blob/master/Setup-TDE-23.md

Any ideas ?

----------------------------------------------------------------------------------------------------------------

[root@sandbox ~]# sudo hadoop dfs -put myfile.txt /zone_encr DEPRECATED: Use of this script to execute hdfs command is deprecated. Instead use the hdfs command for it. put: User:root not allowed to do 'DECRYPT_EEK' on 'key1' 17/01/22 20:32:40 ERROR hdfs.DFSClient: Failed to close inode 43745 org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): No lease on /zone_encr/myfile.txt._COPYING_ (inode 43745): File does not exist. Holder DFSClient_NONMAPREDUCE_-1520880249_1 does not have any open files. at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3439) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:3529) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:3496) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:851) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:536) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145) at org.apache.hadoop.ipc.Client.call(Client.java:1427) at org.apache.hadoop.ipc.Client.call(Client.java:1358) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) at com.sun.proxy.$Proxy9.complete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.complete(ClientNamenodeProtocolTranslatorPB.java:462) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:252) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104) at com.sun.proxy.$Proxy10.complete(Unknown Source) at org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:2358) at org.apache.hadoop.hdfs.DFSOutputStream.closeImpl(DFSOutputStream.java:2340) at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2304) at org.apache.hadoop.hdfs.DFSClient.closeAllFilesBeingWritten(DFSClient.java:951) at org.apache.hadoop.hdfs.DFSClient.closeOutputStreams(DFSClient.java:983) at org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:1086) at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2744) at org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2761) at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)

3 REPLIES 3

Re: HDFS Encryption - error in loading data into encryption zone (org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException)

Expert Contributor

@Ali Bajwa - any ideas on this ?

i'm using this link you wrote - https://github.com/abajwa-hw/security-workshops/blob/master/Setup-TDE-23.md

Highlighted

Re: HDFS Encryption - error in loading data into encryption zone (org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException)

@Karan Alang this is the error:

User:root not allowed to do 'DECRYPT_EEK' on 'key1'

Sounds like you may need to login to Ranger as keyadmin/keyadmin and create a policy that allows DECRYPT_EEK access for user root on the key called key1

Also the guide above was written back in HDP 2.3 timeframe.

For HDP 2.5, you can refer to this guide: https://github.com/HortonworksUniversity/Security_Labs

For HDP 2.4, there is an archive of above guide which can be downloaded here: https://github.com/HortonworksUniversity/Security_Labs/releases/tag/HDP-2.4

Re: HDFS Encryption - error in loading data into encryption zone (org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException)

New Contributor

Still not able to find out resolution the above mentioned

Don't have an account?
Coming from Hortonworks? Activate your account here