Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Sqoop hive import getting failed with --query clause but works fine with --table clause.

Highlighted

Sqoop hive import getting failed with --query clause but works fine with --table clause.

New Contributor

Hello,

 

I am trying to do a hive import from a DB2 table with --query clause as below

 

sqoop import -D db2.jcc.charsetDecoderEncoder=3 \
--driver com.ibm.db2.jcc.DB2Driver \

--connect "jdbc connection string" --username $USER_NM -P

--query "Select * from db.test where col1='111' and $CONDITIONS with ur"

--hive-import \
--hive-table TARGET_TBL\
--hive-partition-key PTN_KEY \
--hive-partition-value PTN_VALUE \

--fields-terminated-by \007 \
--as-textfile

 

When I execute the above command, error :"Must specify destination with --target-dir." is thrown. So, when I change the above command to below, it is working fine.

 

sqoop import -D db2.jcc.charsetDecoderEncoder=3 \
--driver com.ibm.db2.jcc.DB2Driver \

--connect "jdbc connection string" --username $USER_NM -P

--table db.test

--where "col1='111'"

--hive-import \
--hive-table TARGET_TBL\
--hive-partition-key PTN_KEY \
--hive-partition-value PTN_VALUE \

--fields-terminated-by \007 \
--as-textfile

 

Is there any issue with Hive import when we use --query clause? Because the same hive import is working fine if I provide --table clause instead of --query clause.

 

I need to use select query because I have a complex query to be performed at the source table before the import starts.

 

Please explain the issue here.

 

Regards

Harish Kotha.

 

2 REPLIES 2

Re: Sqoop hive import getting failed with --query clause but works fine with --table clause.

New Contributor

I got the same error. I dont know why it is expecting --target-dir while using --query and --hive-import together.

But i resolved it by giving --target-dir argument along with the command. It worked fine with creating new empty directory in the location i gave in target-dir and imported data to the specified hive table.

Re: Sqoop hive import getting failed with --query clause but works fine with --table clause.

Champion

case 1

 

The reason it is asking for --target-dir because during hive import , sqoop will perform hdfs import to a temporary location , once its successful then it will executet the two query internally  . one for creation of table and while other for loading the data

 

case 2

if you provide --table  which already exists s and contains data sqoop will append to the newly imported data.

without the need of temporary location.