Support Questions

Find answers, ask questions, and share your expertise

Sqoop Merge: Can't parse input data: '\N'

avatar
New Contributor

Hi All,

I am getting error at Sqoop merge as below, Sqoop import is working fine for this table.

Can you please help? Any help is appreciated.

Error: java.lang.RuntimeException: Can't parse input data: '\N'
at table.__loadFromFields(table_class.java:1233)
at table.parse(table_class.java:1081)
at org.apache.sqoop.mapreduce.MergeTextMapper.map(MergeTextMapper.java:53)
at org.apache.sqoop.mapreduce.MergeTextMapper.map(MergeTextMapper.java:34)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1714)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.NumberFormatException
at java.math.BigDecimal.<init>(BigDecimal.java:470)
at java.math.BigDecimal.<init>(BigDecimal.java:739)
at table.__loadFromFields(table_class.java:1190)
... 11 more

and Sqoop import command is as follows-

<start to="SQOOP_IMPORT_TABLE" />
<action name="SQOOP_IMPORT_TABLE">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<arg>import</arg>
<arg>-Dmapred.child.java.opts=-Xmx4096m</arg>
<arg>--connect</arg>
<arg>${database}</arg>
<arg>--query</arg>
<arg>${query}</arg>
<arg>--fields-terminated-by</arg>
<arg>'\001'</arg>
<arg>--username</arg>
<arg>${username}</arg>
<arg>--password-file</arg>
<arg>${password}</arg>
<arg>--null-string</arg>
<arg>\\n</arg>
<arg>--null-non-string</arg>
<arg>\\n</arg>
<arg>--target-dir</arg>
<arg>/user/${queueName}/landing/staging/${table}/${PERIOD_DATA_DATE}_uncompressed</arg>
<arg>-m</arg>
<arg>1</arg>
<arg>--map-column-java</arg>
<arg>WKT=String</arg>
<arg>--jar-file</arg>
<arg>${jar_file}</arg>
<arg>--class-name</arg>
<arg>TABLE</arg>
<file>../${conf_location}/${jar_file}#${jar_file}</file>
<file>../${conf_location}/${ojdbc_jar_file}#${ojdbc_jar_file}</file>
</sqoop>
<ok to="DELETE_UNCOMPRESSED_TABLE" />
<error to="Email_failure" />

Thanks,

4 REPLIES 4

avatar
Expert Contributor

It looks like you have \n defined for null-non-string, but in the input it has a null as \N.

Changing the null-non-string to \\N should resolve the issue (unless you have both \n and \N in the source, in which case you need to cleanse your data).

Hopefully this helps!

avatar
New Contributor
I have tried using \\N as well, its still giving same error. Input data from Oracle DB either have some value in it or it is null.

avatar
Expert Contributor

My apologies, I misunderstood the question. You say the import is working for the table but sqoop-merge is failing. Are you running a merge against a dataset that had null defined as \N and one where null was defined as \n?

avatar
New Contributor

running a merge against a dataset that had null defined as \N.