question Re: HDFS Data Recovery in Archives of Support Questions (Read Only)

HDFS Data Recovery

khaslbeck — Fri, 18 Nov 2016 21:23:06 GMT

What are the best recovery options if a product like Abinitio runs an m_rm command that deletes the HDFS data in one of the environments.

These type of low level executions by-pass the Hadoop dfs rm command that puts the deleted data in the trash folder for recover.

The Trash Interval is Configured for 21 Days in the Hortonworks Environment.

Data had to be recreated from the source files, but if this were prod what are the best recovery options?

Re: HDFS Data Recovery

KuldeepK — Fri, 18 Nov 2016 21:42:47 GMT

@Kirk Haslbeck

We can use protected directory feature for production to avoid accidental deletion of data from HDFS

https://issues.apache.org/jira/browse/HDFS-8983

Re: HDFS Data Recovery

khaslbeck — Fri, 18 Nov 2016 22:19:09 GMT

What about Ranger, can that provide protection at this level? Assuming data does get removed any recovery options?

Re: HDFS Data Recovery

sshimpi — Fri, 18 Nov 2016 22:41:52 GMT

@Kirk Haslbeck

Ranger will allow you to authorize centrally where you can allow/disallow access to user level for creating or deleting directories. but it does not provide recovery options.

Pls do check - http://hortonworks.com/apache/ranger/