Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to automatically sync a Hive external table with a MySQL table without using Sqoop?

Highlighted

How to automatically sync a Hive external table with a MySQL table without using Sqoop?

New Contributor

Hi All,

 

I'm already having a MySQL table in my local machine (Linux) itself, and I have a Hive external table with the same schema as the MySQL table.

I want to sync my hive external table whenever new record is inserted or updated.Batch update is ok with me say hourly.
What is the best possible approach to achieve the same.

 

Thanks,

Sumit

1 REPLY 1

Re: How to automatically sync a Hive external table with a MySQL table without using Sqoop?

Guru
Hi Sumit,

Sqoop is an option here, it supports incremental updates, which can remember last update ID if used as saved jobs.

Please refer to below doc:
https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_saved_jobs_and_incremental_imports

And then you can setup cron job to import on hourly basis.

Hope above info helps.