Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Sqoop Merge: Can't parse input data: '\N'

Sqoop Merge: Can't parse input data: '\N'

New Contributor

Hi All,

I am getting error at Sqoop merge as below, Sqoop import is working fine for this table.

Can you please help? Any help is appreciated.

Error: java.lang.RuntimeException: Can't parse input data: '\N'
at table.__loadFromFields(table_class.java:1233)
at table.parse(table_class.java:1081)
at org.apache.sqoop.mapreduce.MergeTextMapper.map(MergeTextMapper.java:53)
at org.apache.sqoop.mapreduce.MergeTextMapper.map(MergeTextMapper.java:34)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1714)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.NumberFormatException
at java.math.BigDecimal.<init>(BigDecimal.java:470)
at java.math.BigDecimal.<init>(BigDecimal.java:739)
at table.__loadFromFields(table_class.java:1190)
... 11 more

and Sqoop import command is as follows-

<start to="SQOOP_IMPORT_TABLE" />
<action name="SQOOP_IMPORT_TABLE">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<arg>import</arg>
<arg>-Dmapred.child.java.opts=-Xmx4096m</arg>
<arg>--connect</arg>
<arg>${database}</arg>
<arg>--query</arg>
<arg>${query}</arg>
<arg>--fields-terminated-by</arg>
<arg>'\001'</arg>
<arg>--username</arg>
<arg>${username}</arg>
<arg>--password-file</arg>
<arg>${password}</arg>
<arg>--null-string</arg>
<arg>\\n</arg>
<arg>--null-non-string</arg>
<arg>\\n</arg>
<arg>--target-dir</arg>
<arg>/user/${queueName}/landing/staging/${table}/${PERIOD_DATA_DATE}_uncompressed</arg>
<arg>-m</arg>
<arg>1</arg>
<arg>--map-column-java</arg>
<arg>WKT=String</arg>
<arg>--jar-file</arg>
<arg>${jar_file}</arg>
<arg>--class-name</arg>
<arg>TABLE</arg>
<file>../${conf_location}/${jar_file}#${jar_file}</file>
<file>../${conf_location}/${ojdbc_jar_file}#${ojdbc_jar_file}</file>
</sqoop>
<ok to="DELETE_UNCOMPRESSED_TABLE" />
<error to="Email_failure" />

Thanks,

4 REPLIES 4

Re: Sqoop Merge: Can't parse input data: '\N'

Rising Star

It looks like you have \n defined for null-non-string, but in the input it has a null as \N.

Changing the null-non-string to \\N should resolve the issue (unless you have both \n and \N in the source, in which case you need to cleanse your data).

Hopefully this helps!

Highlighted

Re: Sqoop Merge: Can't parse input data: '\N'

New Contributor
I have tried using \\N as well, its still giving same error. Input data from Oracle DB either have some value in it or it is null.

Re: Sqoop Merge: Can't parse input data: '\N'

Rising Star

My apologies, I misunderstood the question. You say the import is working for the table but sqoop-merge is failing. Are you running a merge against a dataset that had null defined as \N and one where null was defined as \n?

Re: Sqoop Merge: Can't parse input data: '\N'

New Contributor

running a merge against a dataset that had null defined as \N.

Don't have an account?
Coming from Hortonworks? Activate your account here