Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

HDP Spark HBase Connector

Super Guru

Is there any documentation available on HDP Spark HBase connector?

1 ACCEPTED SOLUTION

Super Guru

I found the documentation (not on HDP at this time)

https://github.com/hortonworks/shc

View solution in original post

15 REPLIES 15

+1

I am looking for the same information. Can someone also share the following pieces of information

  • Sample code using Spark HBase Connector
  • Link to latest documentation
  • Is this GA yet?

Mentor

Spark HBase is in technical preview with Spark 1.6.1 and HDP 2.4.2. http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_spark-guide/content/ch_introduction-spark...

Super Guru

I found the documentation (not on HDP at this time)

https://github.com/hortonworks/shc

Super Guru

has anyone written a demo or app on this one?

New Contributor

Is this technical preview of "Spark HBase Connector" available on latest Sandbox? I believe latest Sandbox is 2.4.0 and technical preview available on 2.4.2? And Github documentation steps to be followed to get this working?

New Contributor

Thanks Sandeep. I will check it out!

Super Guru

check hdp 2.5

Rising Star

Is the spark hbase connector GA in HDP 2.5.3 ?

Expert Contributor

Yes. For Spark 1.6 it is GA in HDP 2.5.3. The documentation is available from the Github site for a given SHC release tag. That is the source of truth.

I try to use in HDP 2.4.0 and cannot run the samples files. I have a Cluster HDP 2.4

mvn clean package test

Cause: java.net.BindException: Address already in use

at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:433)

....

[INFO] HBase Spark Connector Project Parent POM ........... SUCCESS [ 1.542 s] [INFO] HBase Spark Connector Project Core ................. FAILURE [01:04 min] [INFO] HBase Spark Connector Project Examples ............. SKIPPED

New Contributor

hi

is there a way to use the connector offline? is the connector included in HDP 2.6?

that is without using --repositories http://repo.hortonworks.com/content/groups/public/

i am developing in a closed offline environment and cannot download from the internet the connector...

if not, can anyone provide another way of pyspark-hbase connection?

,

hi i'm using python pyspark,

is there a way to run spark-submit offline?

that is without using --repositories http://repo.hortonworks.com/content/groups/public/

is the connector included in HDP 2.6?

Rising Star

I am in the exact same configuration (no way to reach internet from our cluster...), did you find any other option to make it run ?

Super Collaborator