Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

What is the best, most performant, method to join a Hive table with data in Hbase?

Solved Go to solution

What is the best, most performant, method to join a Hive table with data in Hbase?

I have batch data stored in Hive and realtime streaming data stored in HBase. I would like to create a view in Hive which joins a table in Hive with data in HBase. Using Hive on HBase is extremely slow. Is there a better way to accomplish this?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: What is the best, most performant, method to join a Hive table with data in Hbase?

Rising Star

Sort Bucket the Hive table and Read the bucketed Hive table in Mapreduce program and hit Hbase when the Key changes. Requires programming effor, but very effective. Bucketing the Hive table will make sure that a particular key goes to only one bucket, so you hit Hbase Once for a particular key.

View solution in original post

2 REPLIES 2
Highlighted

Re: What is the best, most performant, method to join a Hive table with data in Hbase?

Rising Star

Sort Bucket the Hive table and Read the bucketed Hive table in Mapreduce program and hit Hbase when the Key changes. Requires programming effor, but very effective. Bucketing the Hive table will make sure that a particular key goes to only one bucket, so you hit Hbase Once for a particular key.

View solution in original post

Highlighted

Re: What is the best, most performant, method to join a Hive table with data in Hbase?

@hrongali@hortonworks.com I think a hive UDF could implement same logic, but would be easier to consume than map-reduce program. I think this UDF from brickhouse do this:

https://github.com/klout/brickhouse/blob/master/src/main/java/brickhouse/hbase/CachedGetUDF.java

Don't have an account?
Coming from Hortonworks? Activate your account here