Support Questions

Find answers, ask questions, and share your expertise

Checking hive partition

avatar
Rising Star

Created table in Hive with dynamic partition enabled..

Adding partition on daily basis

ALTER TABLE test ADD PARTITION (date='2014-03-17') location 's3://test.com/2014-03-05'

Now ,How do I ensure that whether partition added or not ?

5 REPLIES 5

avatar
Expert Contributor
show partitions mytable;

Note: if you have more than 500 partitions, you may want to output to a file:

$hive -e 'show partitions mytable;' > partitions

ref: http://stackoverflow.com/questions/15616290/hive-how-to-show-all-partitions-of-a-table

avatar

Refer below:

http://hadooptutorial.info/partitioning-in-hive/#Show_Partitions

For ex:

hive> SHOW PARTITIONS partitioned_user;
OK
country=AU/state=AC
country=AU/state=NS
country=AU/state=NT
country=AU/state=QL
country=AU/state=SA

avatar
Rising Star

Thanks for your answer, I am looking for kind of API to tell that, partition added or not.. I will be difficult to check the partition status daily and take call when there is a failure. ( Want to automate rather checking it manually)

avatar

@Gobi Subramani - How are you creating the partitions? Is there a particular pattern, you will be following.

If yes, you can write a small shell / python script which can call the above command to check if the partition exists. And then you can probably add that to the cron job.

avatar
Rising Star
@Namit Maheshwari

Yes, there is a pattern for creating partition (yyyy-mm-dd) ..

OK,your idea is , run the command and store the result and check for the existence of the partition ?? Is there any other simple way to check ?