Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to automatically sync a MySQL table with a Hive external table using Sqoop?

avatar

I'm already having a MySQL table in my local machine (Linux) it self, and I have a Hive external table with the same schema as the MySQL table.

I'm trying to import data from MySQL table to my Hive external table and I'm using sqoop for this.

But then the problem is, whenever a new record is being added to the MySQL table, it doesn't update the Hive external table automatically?

This is the sqoop import command I'm using:

sqoop import --connect jdbc:mysql://localhost:3306/sqoop --username root -P --split-by id --columns id,name,age,salary --table customer --target-dir /user/chamith/mysqlhivetest/ --fields-terminated-by ","  --hive-import --hive-table test.customers

Am I missing something over here? Or how can this be done?

Any help could be appreciated.

1 ACCEPTED SOLUTION

avatar
@Kulasangar Gowrisangar

You need to save a sqoop job for it to happen automatically.

You can follow this article:

https://dwbi.org/etl/bigdata/195-sqoop-merge-incremental-extraction-from-oracle

View solution in original post

3 REPLIES 3

avatar
@Kulasangar Gowrisangar

You need to save a sqoop job for it to happen automatically.

You can follow this article:

https://dwbi.org/etl/bigdata/195-sqoop-merge-incremental-extraction-from-oracle

avatar

@Kulasangar Gowrisangar

If this worked for you please click the accept button in the answer to close the thread.

Thanks

avatar

Thank you @rbiswas