Support Questions

Find answers, ask questions, and share your expertise

How to perform lookup operation in spark dataframe

avatar
New Member

Based on the columns in spark dataframe need to do a lookup on another huge HBASE table. Is there any efficient way available to perform lookup operation in Spark Dataframe

1 REPLY 1

avatar
Super Collaborator

I think if one of the columns in the dataframe is the key of the HBase table, the lookup will be very efficient; that is, if there is a bottleneck, I don't believe the bottleneck will be the lookup.