Sqoop Incremental update with Merge-key failing with "Error: java.lang.RuntimeException: Can't parse input data:" For the first time job was successful. But since 2nd run it is failing. Below is the sqoop job used.
Sqoop job:
sqoop job -Dhadoop.security.credential.provider.path=jceks://hdfs/user/username/username.password.jceks --create temp_xxxx_update --meta-connect jdbc:hsqldb:hsql://xxxxxx.xx.xxxx.com:16000/sqoop -- import --connect "jdbc:oracle:thin:@yyyy.yyy.yyyy:yyyy" --username username --password-alias PWD --target-dir /hdfs/xxxx/staging/temp_xxxx_update --split-by ID --merge-key ID --table TEMP_xxxx --incremental lastmodified --check-column LAST_MODIFIED_COL --last-value '2017-08-04 05:53:10.0' --input-lines-terminated-by '\n' --input-null-string "\\\\N" --input-null-non-string "\\\\N" --direct --fetch-size 1000 -m 4
Error:
17/08/06 13:50:44 INFO mapreduce.Job: Task Id : attempt_1501783998019_3059_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: Can't parse input data: '5821C57D7-4471-61E9-E040-7DE36F600F65<SYS>2010-03-18 19:10:56<SYS>2017-07-25 18:09:4411D2010-03-18 19:15:25IPMX2010-03-18 19:10:56004OK2010-03-18 19:10:56Material no=<AZ5409>0nullZPE_MATMAS04650MERGE\N4file=../down/ZPE_MATMAS04/20100318/1268953856729_0.zipnull\Nnull'
at TEMP_FSMSG.__loadFromFields(TEMP_FSMSG.java:1706)
at TEMP_FSMSG.parse(TEMP_FSMSG.java:1504)
at org.apache.sqoop.mapreduce.MergeTextMapper.map(MergeTextMapper.java:53)
at org.apache.sqoop.mapreduce.MergeTextMapper.map(MergeTextMapper.java:34)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.NumberFormatException
at java.math.BigDecimal.<init>(BigDecimal.java:550)
at java.math.BigDecimal.<init>(BigDecimal.java:383)
at java.math.BigDecimal.<init>(BigDecimal.java:806)
at TEMP_FSMSG.__loadFromFields(TEMP_FSMSG.java:1553)
... 11 more
