Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

kudu tables migration from one cluster to another cluster with records/data.

Explorer

Hi Team,

 

Is there a way to migrate kudu tables from one cluster to another cluster with data?

 

Thanks.

3 REPLIES 3

Rising Star
The simplest way is to make a copy of the tables using Impala SQL statements or some Spark code.

For example, with Impala you'd use a CTAS (CREATE TABLE AS SELECT) statement. See https://www.cloudera.com/documentation/enterprise/6/latest/topics/impala_create_table.html#create_ta... for details.

Contributor

hi @Harish19 
there is a solution I'm going to test mentioned in 
https://kudu.apache.org/docs/administration.html
and 
https://docs.cloudera.com/cdp/latest/data-migration/topics/cdp-data-migration-restoring-kudu-data.ht...

the main idea is to create a backup with spark
move it with distcp 
then restore your backup

good luck

Contributor

I have tested the backup/restore solution and seems to be working like charm with spark : 

-First, check and record the names as given in the list of the kudu_master (or the primary elected master in case of multi masters ) 
http://Master1:8051/tables 

 

-Download the kudu-backupX.X.jar in case you can't find it in /opt/cloudera/parcels/CDH-X.Xcdh.XX/lib/ and put it there 

-In kuduMasterAddresses you put the name of your Kudu_master or the names of your three masters separated by ','


-Backup : 
sudo -u hdfs spark2-submit --class org.apache.kudu.backup.KuduBackup  /opt/cloudera/parcels/CDH-X.Xcdh.XX/lib/kudu-backup2_2.11-1.13.0.jar --kuduMasterAddresses MASTER1(,MASTER2,..) --rootPath hdfs:///PATH_HDFS
impala::DB.TABLE

-COPY :
sudo -u hdfs hadoop distcp -i - hdfs:///PATH_HDFS/DB.TABLE hdfs://XXX:8020/kudu_backups/

-Restore:

sudo -u hdfs spark2-submit --class org.apache.kudu.backup.KuduRestore /opt/cloudera/parcels/CDH-X.Xcdh.XX/lib/kudu-backup2_2.11-1.13.0.jar --kuduMasterAddresses MASTER1(,MASTER2,..) --rootPath hdfs:///PATH_HDFS impala::DB.TABLE

finally
INVALIDATE METADATA 

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.