Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

kudu tables migration from one cluster to another cluster with records/data.

avatar
Contributor

Hi Team,

 

Is there a way to migrate kudu tables from one cluster to another cluster with data?

 

Thanks.

3 REPLIES 3

avatar
Expert Contributor
The simplest way is to make a copy of the tables using Impala SQL statements or some Spark code.

For example, with Impala you'd use a CTAS (CREATE TABLE AS SELECT) statement. See https://www.cloudera.com/documentation/enterprise/6/latest/topics/impala_create_table.html#create_ta... for details.

avatar
Rising Star

hi @Harish19 
there is a solution I'm going to test mentioned in 
https://kudu.apache.org/docs/administration.html
and 
https://docs.cloudera.com/cdp/latest/data-migration/topics/cdp-data-migration-restoring-kudu-data.ht...

the main idea is to create a backup with spark
move it with distcp 
then restore your backup

good luck

avatar
Rising Star

I have tested the backup/restore solution and seems to be working like charm with spark : 

-First, check and record the names as given in the list of the kudu_master (or the primary elected master in case of multi masters ) 
http://Master1:8051/tables 

 

-Download the kudu-backupX.X.jar in case you can't find it in /opt/cloudera/parcels/CDH-X.Xcdh.XX/lib/ and put it there 

-In kuduMasterAddresses you put the name of your Kudu_master or the names of your three masters separated by ','


-Backup : 
sudo -u hdfs spark2-submit --class org.apache.kudu.backup.KuduBackup  /opt/cloudera/parcels/CDH-X.Xcdh.XX/lib/kudu-backup2_2.11-1.13.0.jar --kuduMasterAddresses MASTER1(,MASTER2,..) --rootPath hdfs:///PATH_HDFS
impala::DB.TABLE

-COPY :
sudo -u hdfs hadoop distcp -i - hdfs:///PATH_HDFS/DB.TABLE hdfs://XXX:8020/kudu_backups/

-Restore:

sudo -u hdfs spark2-submit --class org.apache.kudu.backup.KuduRestore /opt/cloudera/parcels/CDH-X.Xcdh.XX/lib/kudu-backup2_2.11-1.13.0.jar --kuduMasterAddresses MASTER1(,MASTER2,..) --rootPath hdfs:///PATH_HDFS impala::DB.TABLE

finally
INVALIDATE METADATA