Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to perform lookup operation in spark dataframe

avatar

Based on the columns in spark dataframe need to do a lookup on another huge HBASE table. Is there any efficient way available to perform lookup operation in Spark Dataframe

1 REPLY 1

avatar
Super Collaborator

I think if one of the columns in the dataframe is the key of the HBase table, the lookup will be very efficient; that is, if there is a bottleneck, I don't believe the bottleneck will be the lookup.