question Re: How to replicate all hive ddl and database from one cluster to cloud cluster? in Support Questions

How to replicate all hive ddl and database from one cluster to cloud cluster?

ashneesharma88 — Mon, 04 Jan 2016 20:50:27 GMT

Re: How to replicate all hive ddl and database from one cluster to cloud cluster?

KuldeepK — Tue, 05 Jan 2016 23:22:06 GMT

Hey @Ashnee Sharma

Please refer this - https://community.hortonworks.com/questions/4496/how-to-migrate-hive-data-over-to-new-cluster.html

Re: How to replicate all hive ddl and database from one cluster to cloud cluster?

bvellanki — Tue, 02 Feb 2016 06:51:03 GMT

@Ashnee Sharma : Did @Kuldeep Kulkarni's response answer your question? If yes, please accept the answer

Re: How to replicate all hive ddl and database from one cluster to cloud cluster?

nsabharwal — Tue, 02 Feb 2016 08:24:18 GMT

@Ashnee Sharma

Look into this http://falcon.apache.org/Recipes.html

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_data_governance/content/section_mirroring_data_falcon.html

Re: How to replicate all hive ddl and database from one cluster to cloud cluster?

ashneesharma88 — Mon, 22 Feb 2016 20:18:13 GMT

For migrating a hive installation, I did the following things.

1) Install hive in the new hadoop cluster.

2) Transfer the data present in the hive metastore directory (i.e /user/hive/warehouse) to the new hadoop cluster 3) taken the mysql metastore dump. 4) Install mysql in the new hadoop cluster 5) Open the hive mysql-metastore dump using text readers such as notepad, notepad++ etc and search for hdfs://ip-address-old-namenode:port and replace with hdfs://ip-address-new-namenode:port and save it.

Where ip-address-old-namenode is the ipaddress of namenode of old hadoop cluster and ip-address- new-namenode is the ipaddress of namenode of new hadoop cluster.

6) After doing the above steps, restore the editted mysql dump into the mysql of new hadoop cluster. 7) Configure hive as normal and do the hive schema upgradations if needed.

This is a solution that I discovered when I faced the migration issues. I dont know whether any other

standard methods are available.

This worked for me perfectly. 🙂

Re: How to replicate all hive ddl and database from one cluster to cloud cluster?

KuldeepK — Thu, 18 Aug 2016 12:55:49 GMT

@Ashnee Sharma -

are you still having issues with this? Can you accept best answer or provide your own solution?