Welcome to the Cloudera Community

Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Who agreed with this topic

SQOOP issue with teradata connector powered by teradata and Avro

avatar
New Contributor

Hi, I'm trying to use sqoop to import a table from Teradata into Hive, but am having issues.

 

Everything is working ok when importing as textfile.  However, since some of the columns can contain messy data, and I can't reliably choose text separators that won't appear in the source data, I want to use an avro data file.  However, when I add --as-avrodatafile to the end of my sqoop command, I get a NPE, i.e:

 

 

16/07/30 23:33:47 DEBUG manager.SqlManager: Execute getColumnInfoRawQuery : SELECT t.* FROM "jph" AS t WHERE 1=0
16/07/30 23:33:47 DEBUG util.ClassLoaderStack: Restoring classloader: java.net.FactoryURLClassLoader@a652be
16/07/30 23:33:47 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
java.lang.NullPointerException
        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:763)
        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:786)
        at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:289)
        at org.apache.sqoop.manager.SqlManager.getColumnInfo(SqlManager.java:275)
        at org.apache.sqoop.manager.ConnManager.getColumnInfo(ConnManager.java:393)
        at org.apache.sqoop.orm.ClassWriter.getColumnInfo(ClassWriter.java:1854)
        at org.apache.sqoop.orm.AvroSchemaGenerator.generate(AvroSchemaGenerator.java:71)
        at org.apache.sqoop.orm.AvroSchemaGenerator.generate(AvroSchemaGenerator.java:65)
        at com.cloudera.connector.teradata.imports.BaseImportJob.configureInputFormat(BaseImportJob.java:165)
        at com.cloudera.connector.teradata.imports.TableImportJob.configureInputFormat(TableImportJob.java:32)
        at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:258)
        at com.cloudera.connector.teradata.TeradataManager.importTable(TeradataManager.java:273)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:507)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

Any ideas?  AFAIK, the only supported options are text files, or avro file - sequencefiles don't seem to be supported.

 

 

These are the versions:

 

Running Sqoop version: 1.4.6-cdh5.7.
Cloudera Connector Powered by Teradata' on version 1.4c5

 

 

Who agreed with this topic