Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to perform lookup operation in spark dataframe

How to perform lookup operation in spark dataframe

New Contributor

Based on the columns in spark dataframe need to do a lookup on another huge HBASE table. Is there any efficient way available to perform lookup operation in Spark Dataframe

1 REPLY 1
Highlighted

Re: How to perform lookup operation in spark dataframe

Expert Contributor

I think if one of the columns in the dataframe is the key of the HBase table, the lookup will be very efficient; that is, if there is a bottleneck, I don't believe the bottleneck will be the lookup.

Don't have an account?
Coming from Hortonworks? Activate your account here