Support Questions

Find answers, ask questions, and share your expertise

incremental update in hive using sqoop

How to do incremental update when I am loading data from SQL server to hive tables using sqoop without creating extra temporary tables ?

Incremental insert is working using below command.

--incremental append --check-column id --last-value 5

But update is not working using below

--incremental lastmodified --check-column UPDATE_DATE --last-value '2018-07-19 16:14:38'



Do you see any errors/exceptions? Use double quotes for the date in last-value.

--incremental lastmodified --check-column UPDATE_DATE --last-value "2018-07-19 16:14:38"

@Sandeep Nemuri This is also not working. Every time a new record is inserted instead of updating the existing record.

It means the table in hive now has 2 records , both old and new with the same ID

@Deb, I think this is expected. Incremental imports mode can be used to retrieve only rows newer than some previously-imported set of rows.

There is no direct way to achieve the use case you are looking for. Having said that you can refer this document :

Hope this helps.