Member since
06-26-2018
10
Posts
0
Kudos Received
0
Solutions
06-26-2018
02:02 PM
I am running a sqoop job which has a --query statement. This mandates a --target-dir clause, however when I provide a path say /user/xyz/sqoop_import/database/table1. However this path is not honored and instead the data is stored as a hive table in the warehouse folder hdfs://namenode:8020/apps/hive/warehouse/play. Here is the Sqoop Job run: sqoop import \
--connect "jdbc:sqlserver://xx.aa.dd.aa;databaseName=XYZ" \
--connection-manager org.apache.sqoop.manager.SQLServerManager \ --username XXXX \ --password XXXX \
--num-mappers 20 \
--query "select ID,name,x,y,z from TABLE1 where DT between '2018/01/01' and '2018/01/31' AND \$CONDITIONS" \ --split-by id \
--relaxed-isolation \
--target-dir /user/XXXX/sqoop_import/XYZ/2018/TABLE1 \ --fetch-size=100000 \
--hive-import \ --hive-table TABLE1 \ --hive-partition-key Reportdate \
--hive-partition-value Reportdate \
--as-parquetfile \
--compress \
--compression-codec org.apache.hadoop.io.compress.SnappyCodec;
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Sqoop