Created on 04-05-2017 03:11 AM - edited 09-16-2022 04:24 AM
Created table in Hive with dynamic partition enabled..
Adding partition on daily basis
ALTER TABLE test ADD PARTITION (date='2014-03-17') location 's3://test.com/2014-03-05'
Now ,How do I ensure that whether partition added or not ?
Created 04-05-2017 03:16 AM
show partitions mytable;
Note: if you have more than 500 partitions, you may want to output to a file:
$hive -e 'show partitions mytable;' > partitions
ref: http://stackoverflow.com/questions/15616290/hive-how-to-show-all-partitions-of-a-table
Created 04-05-2017 03:19 AM
Refer below:
http://hadooptutorial.info/partitioning-in-hive/#Show_Partitions
For ex:
hive> SHOW PARTITIONS partitioned_user; OK country=AU/state=AC country=AU/state=NS country=AU/state=NT country=AU/state=QL country=AU/state=SA
Created 04-05-2017 03:30 AM
Thanks for your answer, I am looking for kind of API to tell that, partition added or not.. I will be difficult to check the partition status daily and take call when there is a failure. ( Want to automate rather checking it manually)
Created 04-05-2017 03:40 AM
@Gobi Subramani - How are you creating the partitions? Is there a particular pattern, you will be following.
If yes, you can write a small shell / python script which can call the above command to check if the partition exists. And then you can probably add that to the cron job.
Created 04-05-2017 01:14 PM
Yes, there is a pattern for creating partition (yyyy-mm-dd) ..
OK,your idea is , run the command and store the result and check for the existence of the partition ?? Is there any other simple way to check ?