Support Questions

Find answers, ask questions, and share your expertise

Spark 1.3.1 on HDP 2.2.8 accessing Hbase?

avatar
Cloudera Employee

Is access Hbase possible through Spark 1.3.1/SQL though Hive to Hbase? in Spark 1.4 or is it 1.5?

1 ACCEPTED SOLUTION

avatar

There are multiple Spark to HBase connector. See an example on HDP http://stackoverflow.com/questions/25040709/how-to-read-from-hbase-using-spark

But the existing connectors don't leverage HBase for Scans. We are building a more efficient Spark Hbase connector.

View solution in original post

4 REPLIES 4

avatar
Master Mentor

@dgarrison@hortonworks.com

Based on this

and yes,Spark 1.3 and HBase integration started. Link

Our official blog talks about it here

avatar

In case it helps, we had been able to access data from Phoenix as Spark RDD using early version of HDP 2.3 (Spark 1.3). See steps here

avatar

There are multiple Spark to HBase connector. See an example on HDP http://stackoverflow.com/questions/25040709/how-to-read-from-hbase-using-spark

But the existing connectors don't leverage HBase for Scans. We are building a more efficient Spark Hbase connector.

avatar
Explorer

Splice Machine (Open Source) uses Spark 1.6.1 deployed in Yarn.

https://github.com/splicemachine/spliceengine

Might be interesting to you.