Support Questions
Find answers, ask questions, and share your expertise

HDP Spark HBase Connector

Super Guru

Is there any documentation available on HDP Spark HBase connector?

1 ACCEPTED SOLUTION

Super Guru

I found the documentation (not on HDP at this time)

https://github.com/hortonworks/shc

View solution in original post

15 REPLIES 15

+1

I am looking for the same information. Can someone also share the following pieces of information

  • Sample code using Spark HBase Connector
  • Link to latest documentation
  • Is this GA yet?

Mentor

Spark HBase is in technical preview with Spark 1.6.1 and HDP 2.4.2. http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_spark-guide/content/ch_introduction-spark...

Super Guru

I found the documentation (not on HDP at this time)

https://github.com/hortonworks/shc

Super Guru

has anyone written a demo or app on this one?

New Contributor

Is this technical preview of "Spark HBase Connector" available on latest Sandbox? I believe latest Sandbox is 2.4.0 and technical preview available on 2.4.2? And Github documentation steps to be followed to get this working?

New Contributor

Thanks Sandeep. I will check it out!

Super Guru

check hdp 2.5

Rising Star

Is the spark hbase connector GA in HDP 2.5.3 ?

Expert Contributor

Yes. For Spark 1.6 it is GA in HDP 2.5.3. The documentation is available from the Github site for a given SHC release tag. That is the source of truth.

I try to use in HDP 2.4.0 and cannot run the samples files. I have a Cluster HDP 2.4

mvn clean package test

Cause: java.net.BindException: Address already in use

at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:433)

....

[INFO] HBase Spark Connector Project Parent POM ........... SUCCESS [ 1.542 s] [INFO] HBase Spark Connector Project Core ................. FAILURE [01:04 min] [INFO] HBase Spark Connector Project Examples ............. SKIPPED

New Contributor

hi

is there a way to use the connector offline? is the connector included in HDP 2.6?

that is without using --repositories http://repo.hortonworks.com/content/groups/public/

i am developing in a closed offline environment and cannot download from the internet the connector...

if not, can anyone provide another way of pyspark-hbase connection?

,

hi i'm using python pyspark,

is there a way to run spark-submit offline?

that is without using --repositories http://repo.hortonworks.com/content/groups/public/

is the connector included in HDP 2.6?

Rising Star

I am in the exact same configuration (no way to reach internet from our cluster...), did you find any other option to make it run ?

Super Collaborator
; ;