Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

SQOOP issue with teradata connector powered by teradata and Avro

SQOOP issue with teradata connector powered by teradata and Avro

New Contributor

Hi, I'm trying to use sqoop to import a table from Teradata into Hive, but am having issues.

 

Everything is working ok when importing as textfile.  However, since some of the columns can contain messy data, and I can't reliably choose text separators that won't appear in the source data, I want to use an avro data file.  However, when I add --as-avrodatafile to the end of my sqoop command, I get a NPE, i.e:

 

 

16/07/30 23:33:47 DEBUG manager.SqlManager: Execute getColumnInfoRawQuery : SELECT t.* FROM "jph" AS t WHERE 1=0
16/07/30 23:33:47 DEBUG util.ClassLoaderStack: Restoring classloader: java.net.FactoryURLClassLoader@a652be
16/07/30 23:33:47 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
java.lang.NullPointerException
        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:763)
        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:786)
        at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:289)
        at org.apache.sqoop.manager.SqlManager.getColumnInfo(SqlManager.java:275)
        at org.apache.sqoop.manager.ConnManager.getColumnInfo(ConnManager.java:393)
        at org.apache.sqoop.orm.ClassWriter.getColumnInfo(ClassWriter.java:1854)
        at org.apache.sqoop.orm.AvroSchemaGenerator.generate(AvroSchemaGenerator.java:71)
        at org.apache.sqoop.orm.AvroSchemaGenerator.generate(AvroSchemaGenerator.java:65)
        at com.cloudera.connector.teradata.imports.BaseImportJob.configureInputFormat(BaseImportJob.java:165)
        at com.cloudera.connector.teradata.imports.TableImportJob.configureInputFormat(TableImportJob.java:32)
        at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:258)
        at com.cloudera.connector.teradata.TeradataManager.importTable(TeradataManager.java:273)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:507)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

Any ideas?  AFAIK, the only supported options are text files, or avro file - sequencefiles don't seem to be supported.

 

 

These are the versions:

 

Running Sqoop version: 1.4.6-cdh5.7.
Cloudera Connector Powered by Teradata' on version 1.4c5

 

 

2 REPLIES 2

Re: SQOOP issue with teradata connector powered by teradata and Avro

New Contributor
*bump*

Re: SQOOP issue with teradata connector powered by teradata and Avro

Expert Contributor

I have the same problem, import does not work when using --as-avrodatafile. Did you find a solution?