Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Failsafe incremental import with Sqoop

avatar
Contributor

Hi,

If I'm doing an incremental import from SQL Server to HDFS using Sqoop and one (or more) of the mappers fails (for whatever reason, for instance, a String conversion exception), are the data processed by the other mappers going to be written in the HDFS file anyway? Or is the import process not going to copy any data at all?

Thanks a lot in advance.

1 ACCEPTED SOLUTION

avatar
Master Guru

If a Sqoop MR job fails there will be no output. Also, if you are using import jobs, parameters for "append" and "lastmodified" will not be updated. They are updated only after a successful import.

View solution in original post

2 REPLIES 2

avatar
Master Guru

If a Sqoop MR job fails there will be no output. Also, if you are using import jobs, parameters for "append" and "lastmodified" will not be updated. They are updated only after a successful import.

avatar
Contributor

Thanks, Predrag.