https://community.cloudera.com/t5/Archives-of-Support-Questions/Hive-Clusters-Buckets/m-p/148610#M28366 <P>"My understanding so far is that partitioning a table optimises the performance of queries such that rather than performing the query on the entire table it performs the query only on the partition of interest e.g. find employee details where state = NYC. It will just query the NYC partition and return the employee details, correct? These partitions are stored in separate directories/files in HDFS."</P><P>Correct</P><P>"What is a bucket and why would one use them rather than partitions? I take it a bucket and cluster are the same beast just that you use "clusteredby" to create the buckets?"</P><P>You are correct and buckets are essentially files in these partition folders. Every bucket = one file. You can find the reasoning and the uses for them here:</P><P><A href="https://community.hortonworks.com/questions/23103/hive-deciding-the-number-of-buckets.html" target="_blank">https://community.hortonworks.com/questions/23103/hive-deciding-the-number-of-buckets.html</A></P> Sun, 15 May 2016 19:34:05 GMT bleonhardi 2016-05-15T19:34:05Z https://community.cloudera.com/t5/Archives-of-Support-Questions/Hive-Clusters-Buckets/m-p/148609#M28365 <P>Hi,</P><P>I am having difficulty understanding the concept of buckets/clusters in Hive. </P><P>My understanding so far is that partitioning a table optimises the performance of queries such that rather than performing the query on the entire table it performs the query only on the partition of interest e.g. find employee details where state = NYC. It will just query the NYC partition and return the employee details, correct? These partitions are stored in separate directories/files in HDFS. </P><P>What is a bucket and why would one use them rather than partitions? I take it a bucket and cluster are the same beast just that you use "clusteredby" to create the buckets? </P> Sun, 15 May 2016 18:25:21 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Hive-Clusters-Buckets/m-p/148609#M28365 jgarrigan 2016-05-15T18:25:21Z https://community.cloudera.com/t5/Archives-of-Support-Questions/Hive-Clusters-Buckets/m-p/148610#M28366 <P>"My understanding so far is that partitioning a table optimises the performance of queries such that rather than performing the query on the entire table it performs the query only on the partition of interest e.g. find employee details where state = NYC. It will just query the NYC partition and return the employee details, correct? These partitions are stored in separate directories/files in HDFS."</P><P>Correct</P><P>"What is a bucket and why would one use them rather than partitions? I take it a bucket and cluster are the same beast just that you use "clusteredby" to create the buckets?"</P><P>You are correct and buckets are essentially files in these partition folders. Every bucket = one file. You can find the reasoning and the uses for them here:</P><P><A href="https://community.hortonworks.com/questions/23103/hive-deciding-the-number-of-buckets.html" target="_blank">https://community.hortonworks.com/questions/23103/hive-deciding-the-number-of-buckets.html</A></P> Sun, 15 May 2016 19:34:05 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Hive-Clusters-Buckets/m-p/148610#M28366 bleonhardi 2016-05-15T19:34:05Z