HDFS delete directory operation gives error = "cant be moved into an encryption zone"

[#1]

Following are the encryption zones in hdfs

sudo su --c "hdfs crypto -listZones" hdfs

/user/test_user key1

Create a directory in non encrypted zone as test_user

hdfs dfs -mkdir /tmp/dir3/example

Try to delete the directory created again as test_user

hdfs dfs -rm -r /tmp/dir3/example

Failed to move to trash: /tmp/dir3/example can't be moved into an encryption zone

Any help would be appreciated.

Thanks

[#2 · Accepted solution]

@Kshitij Badani - If the user's home directory is encrypted, the user will not be able to delete the file if its not inside its home folder, unless it uses the "-skipTrash" option.

The user should be able to the delete file with "-skipTrash" option.

The problem is that the trash directory for non-encrypted data resides in the user's home directory. Now if user's home is encrypted, un-encrypted data cannot be renamed to this directory, and therefore delete will fail unless used with "-skipTrash".

The trash directory which is in user's home directory, is to ensure quota is correctly calculated and assigned for deleted data. And moving un-encrypted data to EZ is not allowed for security reasons. If user is encrypting it's home directory, they have to use "-skipTrash" to delete un-encrypted data.

Another way to look at it is following: A user who has its home in an EZ should never be creating any important data that is un-encrypted. Please note that deleting encrypted data doesn't have this issue because for encrypted data the trash is within the EZ itself.

Let me know if you have any more doubts. Thanks!

[#3]

@Namit Maheshwari Thanks for a detailed explanation 🙂