Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Sqoop import is failing to import more than 536 columns

Highlighted

Sqoop import is failing to import more than 536 columns

New Contributor

I'm trying to import a table (Oracle DB) which has 706 columns. Sqoop is failing to import if the number of columns is more than 536, getting NPE exception. I'm using Apache Sqoop 1.4.6 (HDP-2.5.3.0-37). Is this a known issue ??

Below is the sqoop command.

sqoop import --connect <connection string > --username XXXXXXXXXX --password XXXXXXXXX --target-dir <hive_table_path> --fields-terminated-by '~' --null-string '\\N' --null-non-string '\\N' -m 1

Error message:

17/04/07 16:46:32 INFO mapreduce.Job: Task Id : attempt_1491519071779_1849_m_000000_1, Status : FAILED

Error: java.lang.NullPointerException

at oracle.jdbc.driver.T4CVarcharAccessor.unmarshalOneRow(T4CVarcharAccessor.java:312)

at oracle.jdbc.driver.T4CTTIrxd.unmarshal(T4CTTIrxd.java:945)

at oracle.jdbc.driver.T4CTTIrxd.unmarshal(T4CTTIrxd.java:865)

at oracle.jdbc.driver.T4C8Oall.readRXD(T4C8Oall.java:790)

at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:403)

at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)

at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)

3 REPLIES 3

Re: Sqoop import is failing to import more than 536 columns

Super Guru
@Amit Kumar Agarwal

Are you able to import any small table using this? Want to make sure your driver is in the classpath.

Re: Sqoop import is failing to import more than 536 columns

New Contributor

Yes, able to ingest any table less than 536 columns using the sqoop command.

Re: Sqoop import is failing to import more than 536 columns

Super Guru
@Amit Kumar Agarwal

Can you increase the number of mappers. Why do you have only one mapper. Can you try with ten mappers? How much total data are you trying to import?