Support Questions
Find answers, ask questions, and share your expertise

What is the best way to replicate Apache Hive LLAP ACID Tables

What is the best way to replicate Apache Hive LLAP ACID Tables

Super Guru

I have a number of tables in Apache Hive LLAP ACID tables with billions of rows and wish to move them to another server.

What options do I have to migrate the data?

I was thinking to copy the data to standard tables, sqoop out then sqoop into the new environment as flat tables then insert into the acid tables. This is a lot of time and data movement and extra storage and extra steps and requires some hacky scripting.

Any better options?

Falcon? Apache NiFi? Flume? Hive replication? DPS? Spark? DistCp? Anything built?

I know full ACID replication is coming in a future version.

I am running HDP 2.6.3.

I looked at:

http://hadoop.apache.org/docs/r2.7.3/hadoop-distcp/DistCp.html

https://cwiki.apache.org/confluence/display/Hive/Replication

https://community.hortonworks.com/articles/110398/mirroring-datasets-between-hadoop-clusters-with-ap...

https://community.hortonworks.com/questions/41104/hive-replication-between-clusters-falcon-based-hiv...

https://cwiki.apache.org/confluence/display/Hive/HiveReplicationDevelopment

https://issues.apache.org/jira/browse/HIVE-18320