Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Snapshotting Apps/Hive/Warehouse

avatar

Are there any concerns with turning on HDFS snapshots on the Apps/Hive/Warehouse directory? I want to make sure the directory is recoverable if someone were to accidentally rmr it (or if anything else happened).

https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsSnapshots.html

1 ACCEPTED SOLUTION

avatar
Master Mentor

@Andrew Watson

Couple of points

1) Please see this. I found this really useful related to snapshots link

2) Hive exports

Personally, I won't enable snapshots for warehouse location and stick with Hive exports. "Performance concern"

View solution in original post

3 REPLIES 3

avatar
Master Mentor

@Andrew Watson

Couple of points

1) Please see this. I found this really useful related to snapshots link

2) Hive exports

Personally, I won't enable snapshots for warehouse location and stick with Hive exports. "Performance concern"

avatar

@Neeraj Sabharwal and @Andrew Watson if you are using hdfs snapshot, it is important to backup hive metastore database as well, because hdfs snapshot only wont be enough without tables/partitions information if user accidentally execute drop tables or drop databases.

avatar
Rising Star

A mistyped hadoop fs -rmr -skipTrash can have catastrophic consequences, which can be protected against with snapshots. What are the performance concerns ?