Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Sqoop with $condition is required if we use query option and Does sqoop supports multiple columns as Modify Key's ?

Sqoop with $condition is required if we use query option and Does sqoop supports multiple columns as Modify Key's ?

I would like to know the Use of $conditon in Sqoop import along with Query because when I am trying to import table along with query it doesn't have/process any data in HDFS files. Below one is the command i used for Import. sqoop import --connect "jdbc:sqlserver://IP Address:Port;database=sample_IL;username=******;password=******" --query 'select STG_D,clo1,clo2,clo3,year,month from COIN where YEAR_R=2015 and MONTH_R=9 and $CONDITIONS' --split-by COIN.STG_D --target-dir /user/myid/hadoop/OPPLAN_Source/COIN_NEW

1 REPLY 1
Highlighted

Re: Sqoop with $condition is required if we use query option and Does sqoop supports multiple columns as Modify Key's ?

Expert Contributor

@Srinivasa Praveen

Please explain mor ein detail what you mean with Multiple Modify Keys?

If you want to import the results of a query in parallel, then each map task will need to execute a copy of the query, with results partitioned by bounding conditions inferred by Sqoop. Your query must include the token $CONDITIONS which each Sqoop process will replace with a unique condition expression. You must also select a splitting column with --split-by.

For example:

$ sqoop import \
  --query 'SELECT a.*, b.* FROM a JOIN b on (a.id == b.id) WHERE $CONDITIONS' \
  --split-by a.id --target-dir /user/foo/joinresults

Alternately, the query can be executed once and imported serially, by specifying a single map task with -m 1:

$ sqoop import \
  --query 'SELECT a.*, b.* FROM a JOIN b on (a.id == b.id) WHERE $CONDITIONS' \
  -m 1 --target-dir /user/foo/joinresults
Don't have an account?
Coming from Hortonworks? Activate your account here