CBO for Hive over hbase

[#1]

Hi,

I have Hive running over HBase table. Is it interesting in this case to analyze the table for the CBO? Or is it only interesting whe nyou have hive with orc file?

Thanks in advance,

Michel

[#2]

Hello Michel

Right now the Hive plan calculation does not reach out to get Hbases stats so currently no added benefit from the Hbase stats. This being said these are questions that are being worked on in different initiatives, so this will likely change in the future.

[#3 · Accepted solution]

@Michel Sumbul

CBO is mainly for optimization decisions which reduces the cost of query execution and is independent of storage formats like ORC. Below is some of decisions based on CBO:

• How to order Join
• What algorithm to use for a given Join
• Should the intermediate result be persisted or should it be recomputed on operator failure.
• The degree of parallelism at any operator (specifically number of reducers to use).
• Semi Join selection

For details, please refer to below link:

https://cwiki.apache.org/confluence/display/Hive/Cost-based+optimization+in+Hive

Thanks and Regards,

Sindhu

[#4]

Thanks for the info. Then what will be your best advice to improve performance of hive over hbase?

[#5]

Hive using MR-over-hbase-snapshots would be a viable solution.

Perform a snapshot in hbase, then use hive to directly read from the underlying HFiles.