Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Problem with incremental append

Problem with incremental append

New Contributor



i'm having problems with incremental append

here's what i do:


first big import:

sqoop import --connect 'jdbc:sqlserver://SERVER;databasename=DATABASE' --username 'USERNAME' --password 'PASSWORD' --query "select * from TABLE  Where \$CONDITIONS" --hive-import --target-dir=/user/hive/warehouse/TABLE --hive-table TABLE --delete-target-dir --split-by KEY


append command:

sqoop import --connect 'jdbc:sqlserver://SERVER;databasename=DATABASE' --username 'USERNAME' --password 'PASSWORD' --query "select * from TABLE  Where \$CONDITIONS" --hive-import --target-dir=/user/hive/warehouse/TABLE --hive-table TABLE --delete-target-dir --split-by KEY --incremental append --check-column KEY


Everything goes right but i don't find new rows in my hive table (same result adding --last-value parameter)


What do i do wrong?


Thank you


Re: Problem with incremental append

Can you try to use "--table TABLE" instead of "--query"?

Also, try to add "--verbose" to see debug messages so that you can see what Sqoop was doing.

If you are doing incremental append, you probably don't want to "--delete-target-dir", since it is --hive-import, you might not need --target-dir either.

You should have --last-value specified when doing incremental imports to start from where you left off last time.