Support Questions
Find answers, ask questions, and share your expertise

what protocol is used for the new spark hbase connector?

Super Guru

what protocol is used for the new spark hbase connector? is spark using the hbase thrift?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: what protocol is used for the new spark hbase connector?

Super Guru

@Sunile Manjee

Integration between Spark and HBase relies simply on HBaseContext which provides HBase configuration to Spark Executor. So, to answer which protocol is used, the answer is simple RPC. Please check following link for more details.

https://hbase.apache.org/book.html#spark

and here is the github link to HBase Spark module.

https://github.com/apache/hbase/tree/master/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark

View solution in original post

2 REPLIES 2

Re: what protocol is used for the new spark hbase connector?

@Sunile Manjee,

Depending on what data your Spark application attempts to access, it uses the relevant JVM HBase client APIs: filters, scans, gets, range-gets, etc.

See the code here.

Re: what protocol is used for the new spark hbase connector?

Super Guru

@Sunile Manjee

Integration between Spark and HBase relies simply on HBaseContext which provides HBase configuration to Spark Executor. So, to answer which protocol is used, the answer is simple RPC. Please check following link for more details.

https://hbase.apache.org/book.html#spark

and here is the github link to HBase Spark module.

https://github.com/apache/hbase/tree/master/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark

View solution in original post