Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

can we match HBase partitions with Impala Partitions?

Solved Go to solution
Highlighted

can we match HBase partitions with Impala Partitions?

Champion Alumni

Hello,

 

Impala/ is having better performances on partitioned tables (if they are big enough).

However, I use Impala on an external HBase table. My HBase table is partitioned.

 

Therefore, I wonder if I can match the HBase partitions with the Impala/Hive ones?

If yes, How can I do this?

 

 

Thank you!

 

GHERMAN Alina
1 ACCEPTED SOLUTION

Accepted Solutions

Re: can we match HBase partitions with Impala Partitions?

Master Collaborator

Hi!

 

HBase tables are range partitioned by the key and Impala will do the equivalent of "partition pruning" for HBase resions if suitable predicates are available in the query. The "PARTITION" concept of HDFS tables is not really applicable to HBase tables. The docs explain this in more detail, including how to interpred Impala's explain plan to see whether region pruning is taking place or not:

 

http://www.cloudera.com/content/www/en-us/documentation/archive/impala/2-x/2-1-x/topics/impala_hbase...

 

Does that answer your question?

3 REPLIES 3

Re: can we match HBase partitions with Impala Partitions?

Master Collaborator

Hi!

 

HBase tables are range partitioned by the key and Impala will do the equivalent of "partition pruning" for HBase resions if suitable predicates are available in the query. The "PARTITION" concept of HDFS tables is not really applicable to HBase tables. The docs explain this in more detail, including how to interpred Impala's explain plan to see whether region pruning is taking place or not:

 

http://www.cloudera.com/content/www/en-us/documentation/archive/impala/2-x/2-1-x/topics/impala_hbase...

 

Does that answer your question?

Re: can we match HBase partitions with Impala Partitions?

Cloudera Employee

The input data set consists of three tables as shown with the following table creation statements in Impala SQL dialect

Re: can we match HBase partitions with Impala Partitions?

Cloudera Employee

Yes. Use a spark-hbase-connector.