Data Migration between Hbase across major versions (1.x to 2.x) across data centers

[#1]

Hi,

We are using a new Datacenter and have to move the data from old cluster to new cluster.

Cluster	Old	New
HDFS	2.7.1.2.3	3.1.1.3.1
Hbase	1.1.2.2.3.2.0	2.0.0.3.1
Zookeeper	3.4.6.2.3	3.4.9.3.1
OS	Debian 7 (Wheezy)	Debian 9 (Stretch)

Since the major versions are different, I only know of one option for data migration: CopyTable. (+ SyncTable)

Are there any challenges to keep note of, since we have different OS.

Is there a way to online replicate the data?

We have around 40 tables and around 1TB of data for each. How many copy table operations can be run parallel?

Thank you

[#2]

Wonderful question, too bad noone answered this. We are handling something similar (upgrading to HDP3.1.5 from HDP2.6.5 in diffrent cluster - HBase 1.1.2 to Hbase 2.0.2). We asked this through Cloudera support and offered us same solution  - copyTable / syncTable but this can't be run from destination cluster (since the source cluster is in current production) so we're looking at snapshots solution, but we still need to identify if is feasible and the challenges implied...

Any inputs from your experience? did you managed to do this migration/upgrade?

Thank You!

[#3]

@ValiD_M Did you find any solution for this?

[#4]

Did anyone found a solution?

[#5]

@Ivoz As this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post. Thanks!

[#6]

Hello All,

This is an older post which had a few recent followup queries. To close the loop, HBase offers multiple Tools to migrate Data from 1 Cluster to another Cluster like Snapshot, Export-Import, HashTable/SyncTable etc. Most of these Tools relies on MapReduce & uses 1 Mapper per Region of the Source Table. All these Tools works without any concerns. The only part of the ask which can't be answered accurately is the Concurrency/Job Configurations/Mapper Memory etc. These details rely on Customer's Environment Setup & the Bandwidth between the 2 Clusters. As such, Customer can run 1 such HBase MR Job & see the Outcome. Accordingly, Fine-Tune is required. 

If any issues are observed while performing the above HBase MR Job, Feel free to post the Q in a Community Post for fellow Community Members to review & share their thoughts. 

Regards, Smarak