Created 04-07-2017 08:55 PM
I'm trying to import a table (Oracle DB) which has 706 columns. Sqoop is failing to import if the number of columns is more than 536, getting NPE exception. I'm using Apache Sqoop 1.4.6 (HDP-2.5.3.0-37). Is this a known issue ??
Below is the sqoop command.
sqoop import --connect <connection string > --username XXXXXXXXXX --password XXXXXXXXX --target-dir <hive_table_path> --fields-terminated-by '~' --null-string '\\N' --null-non-string '\\N' -m 1
Error message:
17/04/07 16:46:32 INFO mapreduce.Job: Task Id : attempt_1491519071779_1849_m_000000_1, Status : FAILED
Error: java.lang.NullPointerException
at oracle.jdbc.driver.T4CVarcharAccessor.unmarshalOneRow(T4CVarcharAccessor.java:312)
at oracle.jdbc.driver.T4CTTIrxd.unmarshal(T4CTTIrxd.java:945)
at oracle.jdbc.driver.T4CTTIrxd.unmarshal(T4CTTIrxd.java:865)
at oracle.jdbc.driver.T4C8Oall.readRXD(T4C8Oall.java:790)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:403)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
Created 04-07-2017 11:12 PM
Are you able to import any small table using this? Want to make sure your driver is in the classpath.
Created 04-07-2017 11:14 PM
Yes, able to ingest any table less than 536 columns using the sqoop command.
Created 04-07-2017 11:30 PM
Can you increase the number of mappers. Why do you have only one mapper. Can you try with ten mappers? How much total data are you trying to import?