Full HBase backup/restore to external appliance

[#1]

I would like to export HBase snapshot data to a backup appliance outside of hdfs for disaster recovery.

I can use ExportSnapshot to export the table to a non hdfs URI, no problem.

But there is no ImportSnapshot - if I try to use ExportSnapshot to bring it back from the external source outside the cluster, using --copy-from <Backup-appliance-URI>/hbase and --copy-to /hbase complains about an unexpected (non hdfs) URI.

If I use DistCp to copy it to a staging area on the hdfs cluster first then use ExportSnapshot to bring it back it works but is undesirable because I need to create a temporary staging area.

I found I was able to use DistCp directly on the exported snapshot and copy it back to hdfs://hbase and restore the table. This looks like a good solution and seems to work, but are there any unexpected problems I can run into? Is this a recommended approach?

[#2 · Accepted solution]

I would only want to make sure the permissions on the file would match the permissions on the appliance, i had issues with Isilon messing up my ACL going back and forth between OneFS and hdfs. Use the same user in both locations, i dont see a problem with distcp approach.

[#3]

Sorry, one related problem, can ExportSnapshot be passed an external .jar and library files using -files and -libjars to support the external filesystem as can be done with DistCp?If not, is there any other way to make ExportSnapshot work if the DistCp operation needs an external .jar and libraries?

[#4]

I never had a use case where I needed to include 3rd party jars in exportsnapshot or distcp. It's a MR job so it should include, check in the options of the command and absolutely paste the command here if you get a solution.

[#5]

Actually, I checked that ExportSnapshot includes the Tools interface, hence it does support -files and -libjars so I think the approach is still good. Sorry, false panic.