https://community.cloudera.com/t5/Archives-of-Support-Questions/what-protocol-is-used-for-the-new-spark-hbase-connector/m-p/124831#M43253 <P><A rel="user" href="https://community.cloudera.com/users/1486/smanjee.html" nodeid="1486">@Sunile Manjee</A> </P><P>Integration between Spark and HBase relies simply on HBaseContext which provides HBase configuration to Spark Executor. So, to answer which protocol is used, the answer is simple RPC. Please check following link for more details.</P><P><A href="https://hbase.apache.org/book.html#spark" target="_blank">https://hbase.apache.org/book.html#spark</A></P><P>and here is the github link to HBase Spark module.</P><P><A href="https://github.com/apache/hbase/tree/master/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark" target="_blank">https://github.com/apache/hbase/tree/master/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark</A></P> Wed, 12 Oct 2016 03:43:30 GMT mqureshi 2016-10-12T03:43:30Z https://community.cloudera.com/t5/Archives-of-Support-Questions/what-protocol-is-used-for-the-new-spark-hbase-connector/m-p/124829#M43251 <P>what protocol is used for the new spark hbase connector? is spark using the hbase thrift?</P> Wed, 12 Oct 2016 03:20:40 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/what-protocol-is-used-for-the-new-spark-hbase-connector/m-p/124829#M43251 sunile_manjee 2016-10-12T03:20:40Z https://community.cloudera.com/t5/Archives-of-Support-Questions/what-protocol-is-used-for-the-new-spark-hbase-connector/m-p/124830#M43252 <P><A rel="user" href="https://community.cloudera.com/users/1486/smanjee.html" nodeid="1486">@Sunile Manjee</A>,</P><P>Depending on what data your Spark application attempts to access, it uses the relevant JVM HBase client APIs: filters, scans, gets, range-gets, etc.</P><P>See the code <A href="https://github.com/hortonworks-spark/shc/tree/master/src/main/scala/org/apache/spark/sql/execution/datasources/hbase">here</A>.</P> Wed, 12 Oct 2016 03:24:57 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/what-protocol-is-used-for-the-new-spark-hbase-connector/m-p/124830#M43252 rgelhausen 2016-10-12T03:24:57Z https://community.cloudera.com/t5/Archives-of-Support-Questions/what-protocol-is-used-for-the-new-spark-hbase-connector/m-p/124831#M43253 <P><A rel="user" href="https://community.cloudera.com/users/1486/smanjee.html" nodeid="1486">@Sunile Manjee</A> </P><P>Integration between Spark and HBase relies simply on HBaseContext which provides HBase configuration to Spark Executor. So, to answer which protocol is used, the answer is simple RPC. Please check following link for more details.</P><P><A href="https://hbase.apache.org/book.html#spark" target="_blank">https://hbase.apache.org/book.html#spark</A></P><P>and here is the github link to HBase Spark module.</P><P><A href="https://github.com/apache/hbase/tree/master/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark" target="_blank">https://github.com/apache/hbase/tree/master/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark</A></P> Wed, 12 Oct 2016 03:43:30 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/what-protocol-is-used-for-the-new-spark-hbase-connector/m-p/124831#M43253 mqureshi 2016-10-12T03:43:30Z