Support Questions

Find answers, ask questions, and share your expertise

Failsafe incremental import with Sqoop

Hi,

If I'm doing an incremental import from SQL Server to HDFS using Sqoop and one (or more) of the mappers fails (for whatever reason, for instance, a String conversion exception), are the data processed by the other mappers going to be written in the HDFS file anyway? Or is the import process not going to copy any data at all?

Thanks a lot in advance.

1 ACCEPTED SOLUTION

If a Sqoop MR job fails there will be no output. Also, if you are using import jobs, parameters for "append" and "lastmodified" will not be updated. They are updated only after a successful import.

View solution in original post

2 REPLIES 2

If a Sqoop MR job fails there will be no output. Also, if you are using import jobs, parameters for "append" and "lastmodified" will not be updated. They are updated only after a successful import.

Thanks, Predrag.