Posts: 72
Registered: ‎01-19-2017

Import and export hbase snapshot from shared location

Hi all,


      With the below steps we are able to copy the hbase data from cluster to another cluster.


  2. bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot SNAPSHOT_NAME -copy-to hdfs://DESTINATION_CLUSTER_ACTIVE_NAMENODE_ADDRESS:8020/hbase -mappers 16
  3. disable "DEST_TABLENAME"
  4. restore_snapshot "SNAPSHOT_NAME"

But the problem is Source cluster does not have access to the destination cluster namenode. To solve this we are planning to have one shared location which mounted in both cluster nodes and using those we are planing to do import and export the snapshot to avoid multiple time copy of the files since the hbase table data size is hug which is around 10TB of data.


Can you please tell me the command to 

  1. export the snapshot to the local mounted shared File location (instead of hdfs location)
  2. restore snapshot from the local  mounted shared file location. (instead of hdfs location)

Thank you,

J.Ganesh Kumar

Posts: 1,894
Kudos: 433
Solutions: 303
Registered: ‎07-31-2013

Re: Import and export hbase snapshot from shared location

If speed is key here, look at getting the connectivity setup between the clusters, as otherwise you're headed down routes of adding duplicate data copy stages.

You can pull down a HDFS export location to disk via a regular 'hadoop fs -put' or via a local 'hadoop distcp' job with a file:///path destination (ensure you don't overwhelm the file-service if this is NFS/etc. by keeping the mapper count low). This will still be slower than just directly sending it to the other cluster's HDFS.