question Re: How to automate sqoop incremental import using sqoop job? in Archives of Support Questions (Read Only)

How to automate sqoop incremental import using sqoop job?

JeevaKrish — Wed, 25 Jan 2017 23:14:38 GMT

Hi there, How to automate sqoop incremental import using sqoop job? As for as I know, sqoop job remembers the last value. If we create a sqoop job like sqoop job --create myjob -- import --connect blah blah.. sqoop exec myjob and automate the job, It will create the job each time it executes. So, we will get job already exists error. So, Is it possible to automate sqoop incremental import using sqoop job? Please let me know your knowledge on this. Thanks in advance.

Re: How to automate sqoop incremental import using sqoop job?

ravi1 — Wed, 25 Jan 2017 23:36:02 GMT

Execute does not create a sqoop job. sqoop job creates the job and you will get an error if you try creating multiple sqoop job with the same job name.

You need to pass --incremental and --check-col on the job to trigger incremental updates.

Re: How to automate sqoop incremental import using sqoop job?

JeevaKrish — Thu, 26 Jan 2017 01:25:24 GMT

Hi Ravi,

Thank you very much for your prompt reply and it helped me a lot. I understand the operation now. Once again thanks