Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Partition is named 2009-12-23 does not mean that it contains all or only data from that date

Partition is named 2009-12-23 does not mean that it contains all or only data from that date

Contributor

Hi All

    i am learning the Hive i find out below hight lighted sentence not i am not able to understand . Ideally partion is to centralize store some data but if the data in that patition not only contain the patitioned data it is meaning less right ?Please help to explain

 

For example, all "US" data from "2009-12-23" is a partition of the page_views table. Note however, that just because a partition is named 2009-12-23 does not mean that it contains all or only data from that date; partitions are named after dates for convenience; it is the user's job to guarantee the relationship between partition name and data content! Partition columns are virtual columns, they are not part of the data itself but are derived on load.

2 REPLIES 2
Highlighted

Re: Partition is named 2009-12-23 does not mean that it contains all or only data from that date

Contributor
I think I am lacking a bit of a context to your question because I am not sure where that text in bold is from.

But generally speaking, the data within the partition is meant to be indicative of value for the partition column. For example, if your table has a partition column of "location", then all data under "Location=US" should belong to that geographical location. Otherwise, the queries involving this partition, generates incorrect result set.

Hope this helps and let me know if you have additional questions. Thanks

Re: Partition is named 2009-12-23 does not mean that it contains all or only data from that date

Contributor

thank you for your reply . my text is from Hive onlime document . so you mean the document is wrong should contain the same value .

https://cwiki.apache.org/confluence/display/Hive/Tutorial

Data Units

  • Partitions: Each Table can have one or more partition Keys which determines how the data is stored. Partitionsapart from being storage unitsalso allow the user to efficiently identify the rows that satisfy a specified criteria; for example, a date_partition of type STRING and country_partition of type STRING. Each unique value of the partition keys defines a partition of the Table. For example, all "US" data from "2009-12-23" is a partition of the page_views table. Therefore, if you run analysis on only the "US" data for 2009-12-23, you can run that query only on the relevant partition of the table, thereby speeding up the analysis significantly. Note however, that just because a partition is named 2009-12-23 does not mean that it contains all or only data from that date; partitions are named after dates for convenience; it is the user's job to guarantee the relationship between partition name and data content! Partition columns are virtual columns, they are not part of the data itself but are derived on load.