Reply
New Contributor
Posts: 1
Registered: ‎08-28-2017

Sqoop Incremental Append

My database has a timestamp column on the basis of which Iam performing my sqoop incremental import with last modified clause.

But

If I am giving last modified check column value as 11am it doesn't retrieves the records which were inserted at 11am it imports records after that.

How do I import the records processed at 11am.

I don't want to have any duplicate records or any missing records.

Highlighted
Champion
Posts: 545
Registered: ‎05-16-2016

Re: Sqoop Incremental Append

its is always recommended to run this as sqoop job so that you will have your last value being recorded automatically. 

 

would you consider performing 

 

--incremental append 

with 

--check-column

 which specifies the column to be examined when determining which rows to import.

 will insert all the new rows based on the last value

Announcements