Reply
New Contributor
Posts: 1
Registered: ‎06-26-2018

How to automatically sync a Hive external table with a MySQL table without using Sqoop?

Hi All,

 

I'm already having a MySQL table in my local machine (Linux) itself, and I have a Hive external table with the same schema as the MySQL table.

I want to sync my hive external table whenever new record is inserted or updated.Batch update is ok with me say hourly.
What is the best possible approach to achieve the same.

 

Thanks,

Sumit

Cloudera Employee
Posts: 376
Registered: ‎03-23-2015

Re: How to automatically sync a Hive external table with a MySQL table without using Sqoop?

Hi Sumit,

Sqoop is an option here, it supports incremental updates, which can remember last update ID if used as saved jobs.

Please refer to below doc:
https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_saved_jobs_and_incremental_imports

And then you can setup cron job to import on hourly basis.

Hope above info helps.

Announcements