I have a hive ORC table with following data , how can I partition it by year and month using hcatalog during SQOOP import ?
what would be my keys and values ?
2016-01-01 00:00:00.0 2016-01-01 00:00:02.0 2016-01-01 00:00:04.0
Hi @Sami Ahmad,
If you created your table with the PARTITION BY clause then the data you insert via sqoop will conform to the partition strategy. Considering your column includes both year and month you may need to stage the data first and then transform it by parsing out the year and month fields to different columns and include those in your partition by clause.
Hope this helps.
it doesn't partition by year it partitions by yyyy-mm-dd . I want to partition the data by year how can I do it with the above records ?
it only takes the yyyy-mm-dd as key value and creates a partion for jan-01 2016.
what I want is to create one partition for the whole year of 2016 not just 1st jan.
--hcatalog-partition-keys txn_process_date --hcatalog-partition-values 2016-01-01