question Failsafe incremental import with Sqoop in Archives of Support Questions (Read Only)

Failsafe incremental import with Sqoop

jaimecarodrigue — Mon, 20 Jun 2016 17:18:52 GMT

Hi,

If I'm doing an incremental import from SQL Server to HDFS using Sqoop and one (or more) of the mappers fails (for whatever reason, for instance, a String conversion exception), are the data processed by the other mappers going to be written in the HDFS file anyway? Or is the import process not going to copy any data at all?

Thanks a lot in advance.

Re: Failsafe incremental import with Sqoop

pminovic — Mon, 20 Jun 2016 17:43:59 GMT

If a Sqoop MR job fails there will be no output. Also, if you are using import jobs, parameters for "append" and "lastmodified" will not be updated. They are updated only after a successful import.

Re: Failsafe incremental import with Sqoop

jaimecarodrigue — Mon, 20 Jun 2016 17:51:44 GMT

Thanks, Predrag.