Support Questions

Find answers, ask questions, and share your expertise

kudu tables migration from one cluster to another cluster with records/data.

avatar
Contributor

Hi Team,

 

Is there a way to migrate kudu tables from one cluster to another cluster with data?

 

Thanks.

3 REPLIES 3

avatar
Expert Contributor
The simplest way is to make a copy of the tables using Impala SQL statements or some Spark code.

For example, with Impala you'd use a CTAS (CREATE TABLE AS SELECT) statement. See https://www.cloudera.com/documentation/enterprise/6/latest/topics/impala_create_table.html#create_ta... for details.

avatar
Rising Star

hi @Harish19 
there is a solution I'm going to test mentioned in 
https://kudu.apache.org/docs/administration.html
and 
https://docs.cloudera.com/cdp/latest/data-migration/topics/cdp-data-migration-restoring-kudu-data.ht...

the main idea is to create a backup with spark
move it with distcp 
then restore your backup

good luck

avatar
Rising Star

I have tested the backup/restore solution and seems to be working like charm with spark : 

-First, check and record the names as given in the list of the kudu_master (or the primary elected master in case of multi masters ) 
http://Master1:8051/tables 

 

-Download the kudu-backupX.X.jar in case you can't find it in /opt/cloudera/parcels/CDH-X.Xcdh.XX/lib/ and put it there 

-In kuduMasterAddresses you put the name of your Kudu_master or the names of your three masters separated by ','


-Backup : 
sudo -u hdfs spark2-submit --class org.apache.kudu.backup.KuduBackup  /opt/cloudera/parcels/CDH-X.Xcdh.XX/lib/kudu-backup2_2.11-1.13.0.jar --kuduMasterAddresses MASTER1(,MASTER2,..) --rootPath hdfs:///PATH_HDFS
impala::DB.TABLE

-COPY :
sudo -u hdfs hadoop distcp -i - hdfs:///PATH_HDFS/DB.TABLE hdfs://XXX:8020/kudu_backups/

-Restore:

sudo -u hdfs spark2-submit --class org.apache.kudu.backup.KuduRestore /opt/cloudera/parcels/CDH-X.Xcdh.XX/lib/kudu-backup2_2.11-1.13.0.jar --kuduMasterAddresses MASTER1(,MASTER2,..) --rootPath hdfs:///PATH_HDFS impala::DB.TABLE

finally
INVALIDATE METADATA