hi we have dev cluster with 5 nodes and prod cluster with 5 nodes boath with hive installed, now i want to migrate partitioned hive tables from dev to prod cluster,
can someone help me how to properly migrate tables and metastore to prod cluster.
Thanks in advance.
You can copy the HDFS files from your dev cluster to prod cluster, then you can re-create the hive table on the prod cluster and then perform a compute statistic for all the metadata like MSCK REPAIR TABLE command. For re-creating the hive tables, you can get the create statement of the table by doing the show create table <table_name> query in your dev cluster.
Following are the high-level steps involved in a Hive migration
Suppose if clusters are Kerberized then you can refer below links for distcp.
Note: There's no need for export because you can directly copy the data from HDFS between both clusters.
Please accept the answer you found most useful