Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how to partion a hive table based on date

how to partion a hive table based on date

Master Collaborator

I have a hive ORC table with following data , how can I partition it by year and month using hcatalog during SQOOP import ?

what would be my keys and values ?

2016-01-01 00:00:00.0
2016-01-01 00:00:02.0
2016-01-01 00:00:04.0
2 REPLIES 2
Highlighted

Re: how to partion a hive table based on date

Hi @Sami Ahmad,

If you created your table with the PARTITION BY clause then the data you insert via sqoop will conform to the partition strategy. Considering your column includes both year and month you may need to stage the data first and then transform it by parsing out the year and month fields to different columns and include those in your partition by clause.

Hope this helps.

Re: how to partion a hive table based on date

Master Collaborator

it doesn't partition by year it partitions by yyyy-mm-dd . I want to partition the data by year how can I do it with the above records ?

it only takes the yyyy-mm-dd as key value and creates a partion for jan-01 2016.

what I want is to create one partition for the whole year of 2016 not just 1st jan.

--hcatalog-partition-keys txn_process_date  --hcatalog-partition-values 2016-01-01  
Don't have an account?
Coming from Hortonworks? Activate your account here