Support Questions

Find answers, ask questions, and share your expertise

CBO for Hive over hbase

Contributor

Hi,

I have Hive running over HBase table. Is it interesting in this case to analyze the table for the CBO? Or is it only interesting whe nyou have hive with orc file?

Thanks in advance,

Michel

1 ACCEPTED SOLUTION

@Michel Sumbul

CBO is mainly for optimization decisions which reduces the cost of query execution and is independent of storage formats like ORC. Below is some of decisions based on CBO:

  • How to order Join
  • What algorithm to use for a given Join
  • Should the intermediate result be persisted or should it be recomputed on operator failure.
  • The degree of parallelism at any operator (specifically number of reducers to use).
  • Semi Join selection

For details, please refer to below link:

https://cwiki.apache.org/confluence/display/Hive/Cost-based+optimization+in+Hive

Thanks and Regards,

Sindhu

View solution in original post

4 REPLIES 4

Hello Michel

Right now the Hive plan calculation does not reach out to get Hbases stats so currently no added benefit from the Hbase stats. This being said these are questions that are being worked on in different initiatives, so this will likely change in the future.

@Michel Sumbul

CBO is mainly for optimization decisions which reduces the cost of query execution and is independent of storage formats like ORC. Below is some of decisions based on CBO:

  • How to order Join
  • What algorithm to use for a given Join
  • Should the intermediate result be persisted or should it be recomputed on operator failure.
  • The degree of parallelism at any operator (specifically number of reducers to use).
  • Semi Join selection

For details, please refer to below link:

https://cwiki.apache.org/confluence/display/Hive/Cost-based+optimization+in+Hive

Thanks and Regards,

Sindhu

Contributor

Thanks for the info. Then what will be your best advice to improve performance of hive over hbase?

Super Collaborator

Hive using MR-over-hbase-snapshots would be a viable solution.

Perform a snapshot in hbase, then use hive to directly read from the underlying HFiles.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.