Snapshotting Apps/Hive/Warehouse

[#1]

Are there any concerns with turning on HDFS snapshots on the Apps/Hive/Warehouse directory? I want to make sure the directory is recoverable if someone were to accidentally rmr it (or if anything else happened).

https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsSnapshots.html

[#2 · Accepted solution]

@Andrew Watson

Couple of points

1) Please see this. I found this really useful related to snapshots link

2) Hive exports

Personally, I won't enable snapshots for warehouse location and stick with Hive exports. "Performance concern"

[#3]

@Neeraj Sabharwal and @Andrew Watson if you are using hdfs snapshot, it is important to backup hive metastore database as well, because hdfs snapshot only wont be enough without tables/partitions information if user accidentally execute drop tables or drop databases.

[#4]

A mistyped hadoop fs -rmr -skipTrash can have catastrophic consequences, which can be protected against with snapshots. What are the performance concerns ?