question Re: Undeletable HDFS Files in Archives of Support Questions (Read Only)

Undeletable HDFS Files

RakeshE — Thu, 23 Feb 2017 14:00:18 GMT

Hi guys,

How can I make a file undeletable in Hdfs. Any suggestions ?

Thanks in advance

Re: Undeletable HDFS Files

csguna — Thu, 23 Feb 2017 14:11:27 GMT

Just put it under the user directory and set the permission just like you we do Linux fs .

Using hadoop fs shell command.

hadoop fs -chown

Usage: hadoop fs -chmod

In addition for backup We can configure HDFS Snapshots point in time file recovery .

https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HdfsSnapshots.html

Re: Undeletable HDFS Files

RakeshE — Thu, 23 Feb 2017 14:28:11 GMT

Thanks csguna for you reply. I need put files that are not deletable even with superuser. Like chattr in Linux file system.

Re: Undeletable HDFS Files

saranvisa — Thu, 23 Feb 2017 16:57:17 GMT

@RakeshE

You can use Access Control List (ACL) to protect your file in HDFS. Pls refer the below links

https://community.cloudera.com/t5/Security-Apache-Sentry/Hadoop-Security-for-beginners/m-p/48576#M174

https://hortonworks.com/blog/hdfs-acls-fine-grained-permissions-hdfs-files-hadoop/

Re: Undeletable HDFS Files

RakeshE — Fri, 24 Feb 2017 04:32:41 GMT

@saranvisa

Superuser(hdfs) can delete any file in hdfs. So all i need is to make an hdfs file that cannot be deleted by anyone even superuser like the way chattr command do in linux. With ACLs i cannot make a file undeletable for all users.

Thanks

Re: Undeletable HDFS Files

Harsh J — Fri, 24 Feb 2017 04:40:54 GMT

There is no immutability attribute feature in HDFS as there is on certain
Linux filesystems. Your best bet is proper access and authentication
control.

Re: Undeletable HDFS Files

RakeshE — Fri, 24 Feb 2017 05:16:15 GMT

Thank you @Harsh J

Re: Undeletable HDFS Files

frisch — Mon, 28 Oct 2019 17:40:20 GMT

Since Hadoop 2.8, it is possible to make a directory protected and so all its files cannot be deleted, using : fs.protected.directories property.

From documentation:

"A comma-separated list of directories which cannot be deleted even by the superuser unless they are empty. This setting can be used to guard important system directories against accidental deletion due to administrator error."

It does not exactly answer the question but it is a possibility.