Support Questions
Find answers, ask questions, and share your expertise

Is there any provision to avoid "null" value in hive table from type-casting while exporting to Teradata?

Is there any provision to avoid "null" value in hive table from type-casting while exporting to Teradata?

New Contributor

I am using teradata-connector for sqoop export of data from hive to teradata. Data consists of records which were filled with "null" while importing to hive from teradata using teradata JDBC driver. This causes typecast exception while exporting back, for columns with types other than String kind of data types.

The sqoop export code is as follows:

## Export using Teradata Connector
sqoop export \
-Dmapred.job.name=Export_To_Teradata \
-Dteradata.db.output.job.type=hive \
-Dteradata.db.output.file.format=orcfile \
-Dteradata.db.output.source.table.schema="$TableSchema" \
-Dteradata.db.output.source.partition.schema="$PartitionSchema" \
-Dteradata.db.output.source.field.name="$ColumnList" \
-libjars "$LIB_JARS" \
--connect "$ConnectionString" \
--connection-manager org.apache.sqoop.teradata.TeradataConnManager \
--username "$UserName" \
--password "$Password" \
--table "$TableName" \
--columns "$ColumnList" \
--export-dir "$SourceDirPath" \
--verbose \
--num-mappers 4
2 REPLIES 2
Highlighted

Re: Is there any provision to avoid "null" value in hive table from type-casting while exporting to Teradata?

Expert Contributor

Take care: the Teradata connector does not support all the Hive types:

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_HortonworksConnectorForTeradata/content/c...

On one project I faced a problem similar to yours (although it was the other way around: I was importing the data from Teradata into Hive). To overcome the above limitation, I had to do it in 2 steps:

1) Sqoop from Teradata and download all the data into HDFS

2) Create an external (text) Hive table on top of those downloaded data

(And then, the classic final step was to have an ORC table to have my final data with an optimal backend).

You might consider having a two steps approach for your export.

Highlighted

Re: Is there any provision to avoid "null" value in hive table from type-casting while exporting to Teradata?

New Contributor

Thank you @Sourygna Luangsay for responding that fast. I was hoping for a straight approach. Now I need to think of alternatives.