Support Questions
Find answers, ask questions, and share your expertise

Sqoop Export Specific Columns Failure

Sqoop Export Specific Columns Failure

Explorer

Trying to export specific columns from Hive Table to Teradata tables. Can we export specific columns not entire Hive table

 

 

getting Unsupported parameter --columns from below export command

 

sqoop export --connect jdbc:teradat://.... --username xxx --password bbb --table table_name

--columns "col1, col2, col3" --m 6 --export-dir /table/location

 

Running Sqoop Version 1.4.7-cdh6.2.1

7 REPLIES 7
Highlighted

Re: Sqoop Export Specific Columns Failure

Expert Contributor

The issue is not with --columns parameter, actually. Problem is the sqoop can't parse the command because it expects "-m" instead of "--m". Remember, when using a short-form parameter that is a single letter, use a single dash, otherwise use double dash. 

 

aakulov_0-1605762483947.png

 

Hope this helps!

 

Highlighted

Re: Sqoop Export Specific Columns Failure

Explorer

tried with -m and --num-mappers still same error

Highlighted

Re: Sqoop Export Specific Columns Failure

Expert Contributor

Your connection string is "teradat" instead of "teradata" - this can also lead to parsing error. 

 

Otherwise, have you tried exporting the entire table, without giving specific columns?

Highlighted

Re: Sqoop Export Specific Columns Failure

Explorer

the export command is correct, just typo when I typed it here

 

Tried exporting entire table: Hive to Teradata

Error: com.teradata.connector.common.exception.connectorException: Index outof boundary

 

both tables in source and destination have exact same columns all varchar in Teradata and string in Hive/Impala

 

 

Highlighted

Re: Sqoop Export Specific Columns Failure

Expert Contributor

Ok, fair enough. There is also a CDH specific connector for Teradata (available here https://www.cloudera.com/downloads/connectors/sqoop/teradata/1-7c6.html). Try that.
The installation and usage guide is here: https://docs.cloudera.com/documentation/other/connectors/teradata/1-x/PDF/Cloudera-Connector-for-Ter...

Highlighted

Re: Sqoop Export Specific Columns Failure

Expert Contributor

You also want to check the structure of your raw data. Specifically, look for any instances where there are extra delimiters (e.g. a string field that includes commas as part of the string). 

Highlighted

Re: Sqoop Export Specific Columns Failure

Explorer

Everything is done as per the installation guide you provided.... still no joy

 

Weird thing is import from Teradata to CDH works seamlessly,

 

Just can't get export working