Reply
JPH
New Contributor
Posts: 2
Registered: ‎07-30-2016

SQOOP issue with teradata connector powered by teradata and Avro

Hi, I'm trying to use sqoop to import a table from Teradata into Hive, but am having issues.

 

Everything is working ok when importing as textfile.  However, since some of the columns can contain messy data, and I can't reliably choose text separators that won't appear in the source data, I want to use an avro data file.  However, when I add --as-avrodatafile to the end of my sqoop command, I get a NPE, i.e:

 

 

16/07/30 23:33:47 DEBUG manager.SqlManager: Execute getColumnInfoRawQuery : SELECT t.* FROM "jph" AS t WHERE 1=0
16/07/30 23:33:47 DEBUG util.ClassLoaderStack: Restoring classloader: java.net.FactoryURLClassLoader@a652be
16/07/30 23:33:47 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
java.lang.NullPointerException
        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:763)
        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:786)
        at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:289)
        at org.apache.sqoop.manager.SqlManager.getColumnInfo(SqlManager.java:275)
        at org.apache.sqoop.manager.ConnManager.getColumnInfo(ConnManager.java:393)
        at org.apache.sqoop.orm.ClassWriter.getColumnInfo(ClassWriter.java:1854)
        at org.apache.sqoop.orm.AvroSchemaGenerator.generate(AvroSchemaGenerator.java:71)
        at org.apache.sqoop.orm.AvroSchemaGenerator.generate(AvroSchemaGenerator.java:65)
        at com.cloudera.connector.teradata.imports.BaseImportJob.configureInputFormat(BaseImportJob.java:165)
        at com.cloudera.connector.teradata.imports.TableImportJob.configureInputFormat(TableImportJob.java:32)
        at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:258)
        at com.cloudera.connector.teradata.TeradataManager.importTable(TeradataManager.java:273)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:507)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

Any ideas?  AFAIK, the only supported options are text files, or avro file - sequencefiles don't seem to be supported.

 

 

These are the versions:

 

Running Sqoop version: 1.4.6-cdh5.7.
Cloudera Connector Powered by Teradata' on version 1.4c5

 

 

JPH
New Contributor
Posts: 2
Registered: ‎07-30-2016

Re: SQOOP issue with teradata connector powered by teradata and Avro

*bump*
Highlighted
Expert Contributor
Posts: 70
Registered: ‎11-24-2017

Re: SQOOP issue with teradata connector powered by teradata and Avro

I have the same problem, import does not work when using --as-avrodatafile. Did you find a solution?

Announcements
New solutions