How to automatically sync a MySQL table with a Hive external table using Sqoop?

[#1]

I'm already having a MySQL table in my local machine (Linux) it self, and I have a Hive external table with the same schema as the MySQL table.

I'm trying to import data from MySQL table to my Hive external table and I'm using sqoop for this.

But then the problem is, whenever a new record is being added to the MySQL table, it doesn't update the Hive external table automatically?

This is the sqoop import command I'm using:

sqoop import --connect jdbc:mysql://localhost:3306/sqoop --username root -P --split-by id --columns id,name,age,salary --table customer --target-dir /user/chamith/mysqlhivetest/ --fields-terminated-by ","  --hive-import --hive-table test.customers

Am I missing something over here? Or how can this be done?

Any help could be appreciated.

[#2 · Accepted solution]

@Kulasangar Gowrisangar

You need to save a sqoop job for it to happen automatically.

You can follow this article:

https://dwbi.org/etl/bigdata/195-sqoop-merge-incremental-extraction-from-oracle

[#3]

@Kulasangar Gowrisangar

If this worked for you please click the accept button in the answer to close the thread.

Thanks

[#4]

Thank you @rbiswas