Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Problem with incremental append

Highlighted

Problem with incremental append

New Contributor

Hi,

 

i'm having problems with incremental append

here's what i do:

 

first big import:

sqoop import --connect 'jdbc:sqlserver://SERVER;databasename=DATABASE' --username 'USERNAME' --password 'PASSWORD' --query "select * from TABLE  Where \$CONDITIONS" --hive-import --target-dir=/user/hive/warehouse/TABLE --hive-table TABLE --delete-target-dir --split-by KEY

 

append command:

sqoop import --connect 'jdbc:sqlserver://SERVER;databasename=DATABASE' --username 'USERNAME' --password 'PASSWORD' --query "select * from TABLE  Where \$CONDITIONS" --hive-import --target-dir=/user/hive/warehouse/TABLE --hive-table TABLE --delete-target-dir --split-by KEY --incremental append --check-column KEY

 

Everything goes right but i don't find new rows in my hive table (same result adding --last-value parameter)

 

What do i do wrong?

 

Thank you

1 REPLY 1

Re: Problem with incremental append

Guru
Can you try to use "--table TABLE" instead of "--query"?

Also, try to add "--verbose" to see debug messages so that you can see what Sqoop was doing.

If you are doing incremental append, you probably don't want to "--delete-target-dir", since it is --hive-import, you might not need --target-dir either.

You should have --last-value specified when doing incremental imports to start from where you left off last time.