Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

Sqoop hive import getting failed with --query clause but works fine with --table clause.

Sqoop hive import getting failed with --query clause but works fine with --table clause.

New Contributor

Hello,

 

I am trying to do a hive import from a DB2 table with --query clause as below

 

sqoop import -D db2.jcc.charsetDecoderEncoder=3 \
--driver com.ibm.db2.jcc.DB2Driver \

--connect "jdbc connection string" --username $USER_NM -P

--query "Select * from db.test where col1='111' and $CONDITIONS with ur"

--hive-import \
--hive-table TARGET_TBL\
--hive-partition-key PTN_KEY \
--hive-partition-value PTN_VALUE \

--fields-terminated-by \007 \
--as-textfile

 

When I execute the above command, error :"Must specify destination with --target-dir." is thrown. So, when I change the above command to below, it is working fine.

 

sqoop import -D db2.jcc.charsetDecoderEncoder=3 \
--driver com.ibm.db2.jcc.DB2Driver \

--connect "jdbc connection string" --username $USER_NM -P

--table db.test

--where "col1='111'"

--hive-import \
--hive-table TARGET_TBL\
--hive-partition-key PTN_KEY \
--hive-partition-value PTN_VALUE \

--fields-terminated-by \007 \
--as-textfile

 

Is there any issue with Hive import when we use --query clause? Because the same hive import is working fine if I provide --table clause instead of --query clause.

 

I need to use select query because I have a complex query to be performed at the source table before the import starts.

 

Please explain the issue here.

 

Regards

Harish Kotha.

 

2 REPLIES 2

Re: Sqoop hive import getting failed with --query clause but works fine with --table clause.

New Contributor

I got the same error. I dont know why it is expecting --target-dir while using --query and --hive-import together.

But i resolved it by giving --target-dir argument along with the command. It worked fine with creating new empty directory in the location i gave in target-dir and imported data to the specified hive table.

Re: Sqoop hive import getting failed with --query clause but works fine with --table clause.

Champion

case 1

 

The reason it is asking for --target-dir because during hive import , sqoop will perform hdfs import to a temporary location , once its successful then it will executet the two query internally  . one for creation of table and while other for loading the data

 

case 2

if you provide --table  which already exists s and contains data sqoop will append to the newly imported data.

without the need of temporary location.