question Re: How to automatically sync a MySQL table with a Hive external table using Sqoop? in Archives of Support Questions (Read Only)

How to automatically sync a MySQL table with a Hive external table using Sqoop?

kulasangar91 — Mon, 17 Jul 2017 18:58:39 GMT

I'm already having a MySQL table in my local machine (Linux) it self, and I have a Hive external table with the same schema as the MySQL table.

I'm trying to import data from MySQL table to my Hive external table and I'm using sqoop for this.

But then the problem is, whenever a new record is being added to the MySQL table, it doesn't update the Hive external table automatically?

This is the sqoop import command I'm using:

sqoop import --connect jdbc:mysql://localhost:3306/sqoop --username root -P --split-by id --columns id,name,age,salary --table customer --target-dir /user/chamith/mysqlhivetest/ --fields-terminated-by ","  --hive-import --hive-table test.customers

Am I missing something over here? Or how can this be done?

Any help could be appreciated.

Re: How to automatically sync a MySQL table with a Hive external table using Sqoop?

rbiswas1 — Mon, 17 Jul 2017 22:18:18 GMT

@Kulasangar Gowrisangar

You need to save a sqoop job for it to happen automatically.

You can follow this article:

https://dwbi.org/etl/bigdata/195-sqoop-merge-incremental-extraction-from-oracle

Re: How to automatically sync a MySQL table with a Hive external table using Sqoop?

rbiswas1 — Wed, 19 Jul 2017 23:41:23 GMT

@Kulasangar Gowrisangar

If this worked for you please click the accept button in the answer to close the thread.

Thanks

Re: How to automatically sync a MySQL table with a Hive external table using Sqoop?

kulasangar91 — Fri, 21 Jul 2017 16:21:07 GMT

Thank you @rbiswas