28806
DISCUSSIONS
102200
MEMBERS
3161
ARTICLES
Created 07-30-2016 03:59 PM
Hi, I'm trying to use sqoop to import a table from Teradata into Hive, but am having issues.
Everything is working ok when importing as textfile. However, since some of the columns can contain messy data, and I can't reliably choose text separators that won't appear in the source data, I want to use an avro data file. However, when I add --as-avrodatafile to the end of my sqoop command, I get a NPE, i.e:
16/07/30 23:33:47 DEBUG manager.SqlManager: Execute getColumnInfoRawQuery : SELECT t.* FROM "jph" AS t WHERE 1=0 16/07/30 23:33:47 DEBUG util.ClassLoaderStack: Restoring classloader: java.net.FactoryURLClassLoader@a652be 16/07/30 23:33:47 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException java.lang.NullPointerException at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:763) at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:786) at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:289) at org.apache.sqoop.manager.SqlManager.getColumnInfo(SqlManager.java:275) at org.apache.sqoop.manager.ConnManager.getColumnInfo(ConnManager.java:393) at org.apache.sqoop.orm.ClassWriter.getColumnInfo(ClassWriter.java:1854) at org.apache.sqoop.orm.AvroSchemaGenerator.generate(AvroSchemaGenerator.java:71) at org.apache.sqoop.orm.AvroSchemaGenerator.generate(AvroSchemaGenerator.java:65) at com.cloudera.connector.teradata.imports.BaseImportJob.configureInputFormat(BaseImportJob.java:165) at com.cloudera.connector.teradata.imports.TableImportJob.configureInputFormat(TableImportJob.java:32) at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:258) at com.cloudera.connector.teradata.TeradataManager.importTable(TeradataManager.java:273) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:507) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615) at org.apache.sqoop.Sqoop.run(Sqoop.java:143) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Any ideas? AFAIK, the only supported options are text files, or avro file - sequencefiles don't seem to be supported.
These are the versions:
Running Sqoop version: 1.4.6-cdh5.7. Cloudera Connector Powered by Teradata' on version 1.4c5