Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st. We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here. Need help or have questions? Drop us a line at [email protected]
Created on 10-08-201604:18 PM - edited 08-17-201909:13 AM
HDFS Snapshots Overview
HDFS Snapshots are read-only point-in-time copies of the
file system. Snapshots can be taken on a subtree of the file system or the
entire file system. Some common use cases of snapshots are data backup,
protection against user errors and disaster recovery.
The implementation of HDFS Snapshots is efficient in the following ways:
1) Snapshot creation is
instantaneous. The cost is
O(1) excluding the inode lookup
time.
2) Additional memory is used
only when modifications are made relative to a snapshot. Memory usage is
O(M),
where
M is the number of modified files/directories.
3) Blocks in datanodes are not copied. The snapshot
files record the block list and the file size.
4) Snapshots do not adversely affect regular HDFS
operations, and there is a minor performance impact from accessing snapshotted data depending on the number of modifications. The snapshot data is computed by subtracting the modifications from the current data (snapshot data = current data – modifications). Also, modifications are recorded in reverse chronological order so
that the current data can be accessed directly.