Table schema is missing on Sqooping data from Teradata to Hive

[#1]

Hi,

Agend:

When I Sqoop data from Teradata to Hive, it should import both data and create adhoc schema same as Teradata schema in my Hive.

Issue:

When I'm trying to Sqoop the data from the Teradata to Hive table, that data is getting successfully sqooped data from Teradata to database directory on HDFS but the is unable able to create the table schema in Hive.

Below is my sqoop command:

sqoop import -libjars  /usr/hdp/current/sqoop-server/lib/ --connect jdbc:teradata://example/Database=ABCDE --create-hive-table --connection-manager org.apache.sqoop.teradata.TeradataConnManager --create-hive-table --username user --password xxxxx --table cricket_location_store --split-by storeid --target-dir /apps/hive/warehouse/user.db/cricket_location_store

Below is the error message I'm getting:

Processor is: 0s16/08/25 15:58:40 ERROR teradata.TeradataSqoopImportHelper:
Exception running Teradata import jobcom.teradata.connector.common.exception.ConnectorException:
Import Hive table's column schema is missing  at
com.teradata.connector.common.tool.ConnectorJobRunner.runJob(ConnectorJobRunner.java:140)  at
com.teradata.connector.common.tool.ConnectorJobRunner.runJob(ConnectorJobRunner.java:56)  at
org.apache.sqoop.teradata.TeradataSqoopImportHelper.runJob(TeradataSqoopImportHelper.java:370)  at
org.apache.sqoop.teradata.TeradataConnManager.importTable(TeradataConnManager.java:504)  at
org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)  at
org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)  at org.apache.sqoop.Sqoop.run(Sqoop.java:148)  at
org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)  at
org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184)  at
org.apache.sqoop.Sqoop.runTool(Sqoop.java:226)  at
org.apache.sqoop.Sqoop.runTool(Sqoop.java:235)  at
org.apache.sqoop.Sqoop.main(Sqoop.java:244)16/08/25 15:58:40 INFO teradata.TeradataSqoopImportHelper:
Teradata import job completed with exit code 116/08/25 15:58:40 ERROR tool.ImportTool: Encountered
IOException running import job: java.io.IOException: Exception running Teradata
import job   at
org.apache.sqoop.teradata.TeradataSqoopImportHelper.runJob(TeradataSqoopImportHelper.java:373)  at
org.apache.sqoop.teradata.TeradataConnManager.importTable(TeradataConnManager.java:504)  at
org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)  at
org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)  at org.apache.sqoop.Sqoop.run(Sqoop.java:148)  at
org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)  at
org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184)  at
org.apache.sqoop.Sqoop.runTool(Sqoop.java:226)  at
org.apache.sqoop.Sqoop.runTool(Sqoop.java:235)  at
org.apache.sqoop.Sqoop.main(Sqoop.java:244)Caused by:
com.teradata.connector.common.exception.ConnectorException: Import Hive table's
column schema is missing  at
com.teradata.connector.common.tool.ConnectorJobRunner.runJob(ConnectorJobRunner.java:140)  at
com.teradata.connector.common.tool.ConnectorJobRunner.runJob(ConnectorJobRunner.java:56)  at
org.apache.sqoop.teradata.TeradataSqoopImportHelper.runJob(TeradataSqoopImportHelper.java:370)  ... 9 more

Any help is highly appreciated and thanks in advance.

[#2]

@Kit Menke

Can you please share all your finding which you have documented?

Thanks in advance.

[#3]

@SBandaru can you create a hive table ahead of time, then run

sqoop import-libjars /usr/hdp/current/sqoop-server/lib/--connect jdbc:teradata://example/Database=ABCDE --connection-manager org.apache.sqoop.teradata.TeradataConnManager --username user --password xxxxx --table cricket_location_store --split-by storeid --target-dir /apps/hive/warehouse/user.db/cricket_location_store

[#4 · Accepted solution]

@SBandaru

Like @Frank Lu mentioned, there are some things that are not supported when using TDCH with Sqoop. For example, creating the schema automatically with "sqoop import" doesn't work.

Instead, use "sqoop create-hive-table" to create the schema first and then do "sqoop import".

For more info see: https://community.hortonworks.com/content/kbentry/53531/importing-data-from-teradata-into-hive.html