I have
some questions around HDFS snapshots which can be used for backup and DR purposes.
How does snapshots help for
Disaster Recovery? What are the best practices around using snapshots for
DR purposes? Especially trying to understand when data is directly stored on HDFS,
Hive data and HBase data
Can a directory be deleted
using hdfs dfs -rmr -skipTrash /data/snapshot-dir? Or is it that all the
snapshots have to be deleted first and then the snapshotting be disabled
before allowing the directory be deleted?
As I understand, no data is
copied for snapshots, but only metadata is maintained for the blocks
added/ modified / deleted. If
that’s the case, just wondering what happens when the comamnd hdfs dfs -rm
/data/snapshot-dir/file1 is run. Will the file be moved to the trash? If
so, will the snapshot maintain the reference to the entry in trash? Will
trach eviction has any impact in this case?
What happens when one of the
sub-directory under the snapshot directory is deleted? For example, if the
command hdfs dfs -rmr -skipTrash /data/sub-dir is run? Can the data be recovered from
snapshots?
Can snapshots be deleted /
archived automatically based on policies, for example time-based? In the
above example, how long will the sub-dir data be maintained in the
snapshot?
How does snapshots work along
with HDFS quotas. For example, assume a directory with a quota of 1GB with
snapshotting enabled. Assume the directory is closer to its full quota and
a user deleted a large file to store some other dataset. Will the new data
be allowed to be saved to the directory or will the operation be stopped
because the quota limits have been exceeded? Apologies
if some of the questions doesn’t make sense. I am still trying to understand
these concepts at a ground level.
... View more
