Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

CBO for Hive over hbase

Solved Go to solution

CBO for Hive over hbase

Contributor

Hi,

I have Hive running over HBase table. Is it interesting in this case to analyze the table for the CBO? Or is it only interesting whe nyou have hive with orc file?

Thanks in advance,

Michel

1 ACCEPTED SOLUTION

Accepted Solutions

Re: CBO for Hive over hbase

@Michel Sumbul

CBO is mainly for optimization decisions which reduces the cost of query execution and is independent of storage formats like ORC. Below is some of decisions based on CBO:

  • How to order Join
  • What algorithm to use for a given Join
  • Should the intermediate result be persisted or should it be recomputed on operator failure.
  • The degree of parallelism at any operator (specifically number of reducers to use).
  • Semi Join selection

For details, please refer to below link:

https://cwiki.apache.org/confluence/display/Hive/Cost-based+optimization+in+Hive

Thanks and Regards,

Sindhu

4 REPLIES 4
Highlighted

Re: CBO for Hive over hbase

Hello Michel

Right now the Hive plan calculation does not reach out to get Hbases stats so currently no added benefit from the Hbase stats. This being said these are questions that are being worked on in different initiatives, so this will likely change in the future.

Re: CBO for Hive over hbase

@Michel Sumbul

CBO is mainly for optimization decisions which reduces the cost of query execution and is independent of storage formats like ORC. Below is some of decisions based on CBO:

  • How to order Join
  • What algorithm to use for a given Join
  • Should the intermediate result be persisted or should it be recomputed on operator failure.
  • The degree of parallelism at any operator (specifically number of reducers to use).
  • Semi Join selection

For details, please refer to below link:

https://cwiki.apache.org/confluence/display/Hive/Cost-based+optimization+in+Hive

Thanks and Regards,

Sindhu

Re: CBO for Hive over hbase

Contributor

Thanks for the info. Then what will be your best advice to improve performance of hive over hbase?

Re: CBO for Hive over hbase

Super Collaborator

Hive using MR-over-hbase-snapshots would be a viable solution.

Perform a snapshot in hbase, then use hive to directly read from the underlying HFiles.