question Re: hbase table copy from one cluster to other in Archives of Support Questions (Read Only)

hbase table copy from one cluster to other

arunpoy — Sat, 17 Sep 2016 16:24:54 GMT

Is there a way to copy hbase( phoenix) tables from one cluster to the other. If so can anyone tell what is the best option?

Re: hbase table copy from one cluster to other

ahadjidj — Sat, 17 Sep 2016 17:49:03 GMT

Hi @ARUN

You have several option to do this:

CopyTable
Export the table, copy the files into a new cluster and Import the table (see documentation section after CopyTable)
In HDP 2.5, there's a new feature of snapshooting. I am not sure if this feature is complete since I didn't try it. There's an open Jira and the backup/restore feature is listed as Tech Preview.

Note that the first two options can have an impact on the RegionServer while the third one has minimal impact.

Re: hbase table copy from one cluster to other

tyu — Sun, 18 Sep 2016 03:09:58 GMT

You can take a look at:

http://hbase.apache.org/book.html#ops.snapshots

Especially:

http://hbase.apache.org/book.html#ops.snapshots.export

The backup / restore feature in HDP 2.5 makes backing up multiple tables easy to operate.

Re: hbase table copy from one cluster to other

ripu — Mon, 19 Sep 2016 17:41:10 GMT

Please see below options and NOTE

NOTE : for both options CopyTable and Export/Import

Since the cluster is up, there is a risk that edits could be missed in the export process.

http://hbase.apache.org/0.94/book/ops_mgt.html#copytable

CopyTable is a utility that can copy part or of all of a table, either to the same cluster or another cluster. The usage is as follows:

$ bin/hbase org.apache.hadoop.hbase.mapreduce.CopyTable [--starttime=X] [--endtime=Y] [--new.name=NEW] [--peer.adr=ADR] tablename

http://hbase.apache.org/0.94/book/ops_mgt.html#export

14.1.7. Export

Export is a utility that will dump the contents of table to HDFS in a sequence file. Invoke via:

$ bin/hbase org.apache.hadoop.hbase.mapreduce.Export <tablename> <outputdir> [<versions> [<starttime> [<endtime>]]]

Note: caching for the input Scan is configured via hbase.client.scanner.caching in the job configuration.

14.1.8. Import

Import is a utility that will load data that has been exported back into HBase. Invoke via:

$ bin/hbase org.apache.hadoop.hbase.mapreduce.Import <tablename> <inputdir>

Re: hbase table copy from one cluster to other

nitinshk77 — Mon, 19 Sep 2016 18:18:08 GMT

@ARUN,

Both the mathods "Copytable" and "Import/Export of table" are good for this but they will degrade the performance of regionserver while copying. I would preffer "Snapshot" mathod best for Backup and Recovery.

Note:- Snapshot method will only work if both cluster are of same version of Hbase. I tried it.

If your both cluster hbase versions are different then you can use Copytable method.

Snapshot method,

Go to hbase-shell and Take a snapshot of table,

=>hbase shell

=>snapshot "SOURCE_TABLE_NAME","SNAPSHOT_TABLE_NAME"

Then you can Export that snapshot to other cluster like,

=>bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot SNAPSHOT_TABLE_NAME -copy-to hdfs://DESTINATION_CLUSTER_ACTIVE_NAMENODE_ADDRESS:8020/hbase -mappers 16

After this you can restore the table on DESTINATION Cluster as,On Dest_Cluster,

=>hbase shell

=>disable "DEST_TABLENAME"

=>restore_snapshot "SNAPSHOT_TABLE_NAME"

Done your table will be copied.

Re: hbase table copy from one cluster to other

nitinshk77 — Mon, 19 Sep 2016 20:04:18 GMT

@AMIT,

Before using all the methods please take a backup of destination clusters Table by using Snapshot method like,

On destination cluster,

=>hbase shell

=>snapshot "DEST_TABLE_NAME","SNAPSHOT_DEST_TABLE_NAME"

So that your data on DESTINATION cluster will not be lost. To keep your data safe on Destination clutser you can use this this method. After your use you can revert it back as,

=>hbase shell

=>disable "DEST_TABLE_NAME"

=>restore_snapshot "SNAPSHOT_DEST_TABLE_NAME"

Re: hbase table copy from one cluster to other

priyankaajindal — Tue, 30 Jan 2018 23:34:13 GMT

In this if we copy the Hfiles manually from one Hbase cluster to another, in that case list command dispalys all the tables, But scanning a table does not shown any data. This is because i have not copied META table enteries. So is there a way to copy META table enteries to another hbase instance in a way that both the already exisiting table and new tables exist with their data.

If we manually copy Hfiles from Hbase instance to another. In that case, list will display the tables but scan will not show the data because we have not copied META table. So is there a way to copy the enteries of MEta table also, in a way that new tables and already exisiting tables both are retained.

Re: hbase table copy from one cluster to other

priyankaajindal — Tue, 30 Jan 2018 23:34:15 GMT