HDP Spark HBase Connector

[#1]

Is there any documentation available on HDP Spark HBase connector?

[#2]

+1

I am looking for the same information. Can someone also share the following pieces of information

• Sample code using Spark HBase Connector
• Link to latest documentation
• Is this GA yet?

[#3]

Spark HBase is in technical preview with Spark 1.6.1 and HDP 2.4.2. http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_spark-guide/content/ch_introduction-spark...

[#4 · Accepted solution]

I found the documentation (not on HDP at this time)

https://github.com/hortonworks/shc

[#5]

Checkout latest blog on HBase connector: http://hortonworks.com/blog/spark-hbase-dataframe-based-hbase-connector/

[#6]

has anyone written a demo or app on this one?

[#7]

Is this technical preview of "Spark HBase Connector" available on latest Sandbox? I believe latest Sandbox is 2.4.0 and technical preview available on 2.4.2? And Github documentation steps to be followed to get this working?

[#8]

@Venkat Ankam Here is the latest HDP 2.5 sandbox : https://hortonworks.com/tech-preview-hdp-2-5/

Steps to follow are at https://github.com/hortonworks/tutorials/blob/hdp-2.5/tutorials/hortonworks/spark-hbase-a-dataframe-...

[#9]

Thanks Sandeep. I will check it out!

[#10]

check hdp 2.5

[#11]

Is the spark hbase connector GA in HDP 2.5.3 ?

[#12]

Yes. For Spark 1.6 it is GA in HDP 2.5.3. The documentation is available from the Github site for a given SHC release tag. That is the source of truth.

[#13]

I try to use in HDP 2.4.0 and cannot run the samples files. I have a Cluster HDP 2.4

mvn clean package test

Cause: java.net.BindException: Address already in use

at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:433)

....

[INFO] HBase Spark Connector Project Parent POM ........... SUCCESS [ 1.542 s] [INFO] HBase Spark Connector Project Core ................. FAILURE [01:04 min] [INFO] HBase Spark Connector Project Examples ............. SKIPPED

[#14]

hi

is there a way to use the connector offline? is the connector included in HDP 2.6?

that is without using --repositories http://repo.hortonworks.com/content/groups/public/

i am developing in a closed offline environment and cannot download from the internet the connector...

if not, can anyone provide another way of pyspark-hbase connection?

,

hi i'm using python pyspark,

is there a way to run spark-submit offline?

that is without using --repositories http://repo.hortonworks.com/content/groups/public/

is the connector included in HDP 2.6?

[#15]

I am in the exact same configuration (no way to reach internet from our cluster...), did you find any other option to make it run ?

[#16]

Please take a look at:

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_spark-component-guide/content/spark-on-h...