question Re: Snapshotting Apps/Hive/Warehouse in Archives of Support Questions (Read Only)

Snapshotting Apps/Hive/Warehouse

awatson — Thu, 19 Nov 2015 11:20:11 GMT

Are there any concerns with turning on HDFS snapshots on the Apps/Hive/Warehouse directory? I want to make sure the directory is recoverable if someone were to accidentally rmr it (or if anything else happened).

https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsSnapshots.html

Re: Snapshotting Apps/Hive/Warehouse

nsabharwal — Thu, 19 Nov 2015 19:11:47 GMT

@Andrew Watson

Couple of points

1) Please see this. I found this really useful related to snapshots link

2) Hive exports

Personally, I won't enable snapshots for warehouse location and stick with Hive exports. "Performance concern"

Re: Snapshotting Apps/Hive/Warehouse

gbraccialli3 — Thu, 19 Nov 2015 22:43:08 GMT

@Neeraj Sabharwal and @Andrew Watson if you are using hdfs snapshot, it is important to backup hive metastore database as well, because hdfs snapshot only wont be enough without tables/partitions information if user accidentally execute drop tables or drop databases.

Re: Snapshotting Apps/Hive/Warehouse

jpp — Sat, 21 Nov 2015 06:03:34 GMT

A mistyped hadoop fs -rmr -skipTrash can have catastrophic consequences, which can be protected against with snapshots. What are the performance concerns ?