Support Questions

Find answers, ask questions, and share your expertise

How to perform lookup operation in spark dataframe

Based on the columns in spark dataframe need to do a lookup on another huge HBASE table. Is there any efficient way available to perform lookup operation in Spark Dataframe


Expert Contributor

I think if one of the columns in the dataframe is the key of the HBase table, the lookup will be very efficient; that is, if there is a bottleneck, I don't believe the bottleneck will be the lookup.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.