Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Sqoop Incremental update with Merge-key failing with "Error: java.lang.RuntimeException: Can't parse input data:"

Sqoop Incremental update with Merge-key failing with "Error: java.lang.RuntimeException: Can't parse input data:"

New Contributor

Sqoop Incremental update with Merge-key failing with "Error: java.lang.RuntimeException: Can't parse input data:" For the first time job was successful. But since 2nd run it is failing. Below is the sqoop job used.

Sqoop job:

sqoop job -Dhadoop.security.credential.provider.path=jceks://hdfs/user/username/username.password.jceks --create temp_xxxx_update --meta-connect jdbc:hsqldb:hsql://xxxxxx.xx.xxxx.com:16000/sqoop -- import --connect "jdbc:oracle:thin:@yyyy.yyy.yyyy:yyyy" --username username --password-alias PWD --target-dir /hdfs/xxxx/staging/temp_xxxx_update --split-by ID --merge-key ID --table TEMP_xxxx --incremental lastmodified --check-column LAST_MODIFIED_COL --last-value '2017-08-04 05:53:10.0' --input-lines-terminated-by '\n' --input-null-string "\\\\N" --input-null-non-string "\\\\N" --direct --fetch-size 1000 -m 4

Error:

17/08/06 13:50:44 INFO mapreduce.Job: Task Id : attempt_1501783998019_3059_m_000000_0, Status : FAILED Error: java.lang.RuntimeException: Can't parse input data: '5821C57D7-4471-61E9-E040-7DE36F600F65<SYS>2010-03-18 19:10:56<SYS>2017-07-25 18:09:4411D2010-03-18 19:15:25IPMX2010-03-18 19:10:56004OK2010-03-18 19:10:56Material no=<AZ5409>0nullZPE_MATMAS04650MERGE\N4file=../down/ZPE_MATMAS04/20100318/1268953856729_0.zipnull\Nnull' at TEMP_FSMSG.__loadFromFields(TEMP_FSMSG.java:1706) at TEMP_FSMSG.parse(TEMP_FSMSG.java:1504) at org.apache.sqoop.mapreduce.MergeTextMapper.map(MergeTextMapper.java:53) at org.apache.sqoop.mapreduce.MergeTextMapper.map(MergeTextMapper.java:34) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162) Caused by: java.lang.NumberFormatException at java.math.BigDecimal.<init>(BigDecimal.java:550) at java.math.BigDecimal.<init>(BigDecimal.java:383) at java.math.BigDecimal.<init>(BigDecimal.java:806) at TEMP_FSMSG.__loadFromFields(TEMP_FSMSG.java:1553) ... 11 more

1 REPLY 1
Highlighted

Re: Sqoop Incremental update with Merge-key failing with "Error: java.lang.RuntimeException: Can't parse input data:"

Cloudera Employee

Hi Gayathri,

Seems sqoop/hive unable to determine the column delimiter. Can we import the same to HDFS and check the field terminator used in Oracle?

We can try the same with option --fields-terminated-by.

As the first import was successful, can we check the hive table if data is correctly split into corresponding columns?

Don't have an account?
Coming from Hortonworks? Activate your account here