Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can I migrate an entire relational schema into a hadoop cluster using sqoop or any other utility?

Can I migrate an entire relational schema into a hadoop cluster using sqoop or any other utility?

Explorer

 

Hi Everyone,

 

 We are about to set up a new Cloudera Hadoop Cluster and I wonder if it is posible to migrate/move an entire/complete relational schema into a hadoop cluster using sqoop or any other utility?. 

 

 The plan is to create tables in hive so we can mantain the same relations but under a hadoop cluster...

 Is this the beast approach and can be done? 

 

 Thanks and I really appreciate your feedback...

 

 

 

2 REPLIES 2

Re: Can I migrate an entire relational schema into a hadoop cluster using sqoop or any other utility

Champion

Yes you can , make sure you tune some of the parameters  across the clusters after considering the amount of load  you going ship . 

 

for example  - parallelism 

sqoop import-all-tables

Re: Can I migrate an entire relational schema into a hadoop cluster using sqoop or any other utility

Explorer
Thanks for your reply. What about replicating the structure/columns of the tables (DDL in the relational world). I have been copying tables 1 by1 using a process something like:
COPY t1.claim TO '/data/temp/claim.csv' DELIMITER ',' CSV HEADER; ## postgresql command
Then copying the file into the HDFS and then using Hive creating a table and it works like a charm.. The only issue is a very slow process. So you think the import-all-tables option will handle structure/ddl as well.

Thanks again...