Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to automatically sync a MySQL table with a Hive external table using Sqoop?

Solved Go to solution
Highlighted

How to automatically sync a MySQL table with a Hive external table using Sqoop?

I'm already having a MySQL table in my local machine (Linux) it self, and I have a Hive external table with the same schema as the MySQL table.

I'm trying to import data from MySQL table to my Hive external table and I'm using sqoop for this.

But then the problem is, whenever a new record is being added to the MySQL table, it doesn't update the Hive external table automatically?

This is the sqoop import command I'm using:

sqoop import --connect jdbc:mysql://localhost:3306/sqoop --username root -P --split-by id --columns id,name,age,salary --table customer --target-dir /user/chamith/mysqlhivetest/ --fields-terminated-by ","  --hive-import --hive-table test.customers

Am I missing something over here? Or how can this be done?

Any help could be appreciated.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: How to automatically sync a MySQL table with a Hive external table using Sqoop?

@Kulasangar Gowrisangar

You need to save a sqoop job for it to happen automatically.

You can follow this article:

https://dwbi.org/etl/bigdata/195-sqoop-merge-incremental-extraction-from-oracle

View solution in original post

3 REPLIES 3
Highlighted

Re: How to automatically sync a MySQL table with a Hive external table using Sqoop?

@Kulasangar Gowrisangar

You need to save a sqoop job for it to happen automatically.

You can follow this article:

https://dwbi.org/etl/bigdata/195-sqoop-merge-incremental-extraction-from-oracle

View solution in original post

Highlighted

Re: How to automatically sync a MySQL table with a Hive external table using Sqoop?

@Kulasangar Gowrisangar

If this worked for you please click the accept button in the answer to close the thread.

Thanks

Highlighted

Re: How to automatically sync a MySQL table with a Hive external table using Sqoop?

Thank you @rbiswas

Don't have an account?
Coming from Hortonworks? Activate your account here