Support Questions
Find answers, ask questions, and share your expertise

Creating a paritioned hive table using sqoop

Creating a paritioned hive table using sqoop


Hello guys!


 I need a hand finding what is wrong with my sqoop command, I want to create a table partitioned by year, but something is still wrong and I'm not sure why.




sqoop import \ \
--connection-manager org.apache.sqoop.manager.SQLServerManager \
--driver net.sourceforge.jtds.jdbc.Driver \
--connect 'jdbc:jtds:sqlserver://SQLQP002:1433;useNTLMv2=true;domain=JNL_NT;databaseName=TC31Reporting' \
--username 'lrm0613' \
--password-alias sqlserver2.password \
--query 'select jobhistoryid, jobid,requeueid,name,event,eventtime,submittime,starttime,service,year(EventTime) as year from JobHistory where year(EventTime)=2019 and $CONDITIONS' \
--hcatalog-database dataengsandbox \
--hcatalog-table JobHistoryPartitioned \
--hive-partition-key YEAR \
--hive-partition-value 2019 \
--compress \
--compression-codec snappy \
--map-column-hive YEAR=INT \
-m 1 \
--create-hcatalog-table \
--hcatalog-storage-stanza 'stored as parquet'




20/05/06 15:42:21 INFO hcat.SqoopHCatUtilities: Executing external HCatalog CLI process with args :-f,/tmp/hcat-script-1588794141852
20/05/06 15:42:28 INFO hcat.SqoopHCatUtilities: FAILED: SemanticException [Error 10035]: Column repeated in partitioning columns
20/05/06 15:42:29 ERROR tool.ImportTool: Import failed: HCat exited with status 51


Thank you in advance.