Created 01-10-2018 11:01 AM
I read from sqoop documentation
http://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_hcatalog_background
The Sqoop HCatalog feature supports the following table types:
But I am not getting how to use dynamic partition using sqoop.
sqoop import --connect jdbc:mysql://localhost:3306/test --username root --password-file file:///home/hdfs/.password --split-by id --table test --delete-target-dir --fields-terminated-by "," --hcatalog-table test --hcatalog-storage-stanza "stored as orcfile" --create-hcatalog-table --hcatalog-partition-keys created_at --map-column-hive created_at=date,updated_at=date
Getting This error
Either both --hcatalog-partition-keys and --hcatalog-partition-values should be provided or both of these options should be omitted.
and if I use the partition value then it assigns it as static partition
sqoop import --connect jdbc:mysql://localhost:3306/test --username root --password-file file:///home/hdfs/.password --split-by id --table test --delete-target-dir --fields-terminated-by "," --hcatalog-table test --hcatalog-storage-stanza "stored as orcfile" --create-hcatalog-table --hcatalog-partition-keys created_at --hcatalog-partition-value created_at --map-column-hive created_at=date,updated_at=date
after giving created_at as hcatalog-partition-value it created the directory like apps/hive/warehouse/test/created_at=created_at
it should have used created_at column to create dynamic partition based on data.
Please help me fix this issue