Member since
01-10-2018
2
Posts
0
Kudos Received
0
Solutions
01-17-2018
07:20 AM
I read from sqoop documentation http://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_hcatalog_background 23.6. Support for Partitioning The Sqoop HCatalog feature supports the following table types:
Unpartitioned tables Partitioned tables with a static partitioning key specified Partitioned tables with dynamic partition keys from the database result set Partitioned tables with a combination of a static key and additional dynamic partitioning keys But I am not getting how to use dynamic partition using sqoop. sqoop import --connect jdbc:mysql://localhost:3306/test
--username root
--password-file file:///home/hdfs/.password
--split-by id
--table test
--delete-target-dir
--fields-terminated-by ","
--hcatalog-table test
--hcatalog-storage-stanza "stored as orcfile"
--create-hcatalog-table
--hcatalog-partition-keys created_at
--map-column-hive created_at=date,updated_at=date Getting This error Either both --hcatalog-partition-keys and --hcatalog-partition-values should be provided or both of these options should be omitted. and if I use the partition value then it assigns it as static partition sqoop import --connect jdbc:mysql://localhost:3306/test
--username root
--password-file file:///home/hdfs/.password
--split-by id
--table test
--delete-target-dir
--fields-terminated-by ","
--hcatalog-table test
--hcatalog-storage-stanza "stored as orcfile"
--create-hcatalog-table
--hcatalog-partition-keys created_at
--hcatalog-partition-value created_at
--map-column-hive created_at=date,updated_at=date
after giving created_at as hcatalog-partition-value it created the directory like apps/hive/warehouse/test/created_at=created_at it should have used created_at column to create dynamic partition based on data. Please help me fix this issue
... View more
Labels:
- Labels:
-
Apache HCatalog
-
Apache Hive
-
Apache Sqoop