Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Who Me Too'd this solution

Re: How to distribute impala table partitions

Master Collaborator

Impala does not have control of the physical locations of the HDFS blocks underlying Impala tables.

The tables in Impala are backed by files on HDFS and those files are chopped into blocks and distributed according to your HDFS configuration, but for all practical purposes the blocks are distributed round-robin among the data nodes (grossly simplified). Impala queries typically run on all data nodes that store data relevant to answering a parcitular query, so given a fixed amount of data, you can indirectly control Impala's degree of (inter-node) parallelism by changing the HDFS block size. More blocks == more parallelism.

If you are interested in learning about Impala, you may also find the CIDR paper useful:
http://www.cidrdb.org/cidr2015/Papers/CIDR15_Paper28.pdf

Who Me Too'd this solution