Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Creating a paritioned hive table using sqoop

Highlighted

Creating a paritioned hive table using sqoop

Contributor

Hello guys!

 

 I need a hand finding what is wrong with my sqoop command, I want to create a table partitioned by year, but something is still wrong and I'm not sure why.

 

Command:

 

sqoop import \
-Dhadoop.security.credential.provider.path=jceks://hdfs/user/lrm0613/mydb2.jceks \
--connection-manager org.apache.sqoop.manager.SQLServerManager \
--driver net.sourceforge.jtds.jdbc.Driver \
--connect 'jdbc:jtds:sqlserver://SQLQP002:1433;useNTLMv2=true;domain=JNL_NT;databaseName=TC31Reporting' \
--username 'lrm0613' \
--password-alias sqlserver2.password \
--query 'select jobhistoryid, jobid,requeueid,name,event,eventtime,submittime,starttime,service,year(EventTime) as year from JobHistory where year(EventTime)=2019 and $CONDITIONS' \
--hcatalog-database dataengsandbox \
--hcatalog-table JobHistoryPartitioned \
--hive-partition-key YEAR \
--hive-partition-value 2019 \
--compress \
--compression-codec snappy \
--map-column-hive YEAR=INT \
-m 1 \
--create-hcatalog-table \
--hcatalog-storage-stanza 'stored as parquet'

 

 

Result:

20/05/06 15:42:21 INFO hcat.SqoopHCatUtilities: Executing external HCatalog CLI process with args :-f,/tmp/hcat-script-1588794141852
20/05/06 15:42:28 INFO hcat.SqoopHCatUtilities: FAILED: SemanticException [Error 10035]: Column repeated in partitioning columns
20/05/06 15:42:29 ERROR tool.ImportTool: Import failed: java.io.IOException: HCat exited with status 51

 

Thank you in advance.

Don't have an account?
Coming from Hortonworks? Activate your account here