question Re: HDP Spark HBase Connector in Support Questions

HDP Spark HBase Connector

sunile_manjee — Wed, 18 May 2016 08:34:09 GMT

Is there any documentation available on HDP Spark HBase connector?

Re: HDP Spark HBase Connector

bsaini — Wed, 18 May 2016 08:34:10 GMT

I am looking for the same information. Can someone also share the following pieces of information

Sample code using Spark HBase Connector
Link to latest documentation
Is this GA yet?

Re: HDP Spark HBase Connector

aervits — Wed, 18 May 2016 08:38:45 GMT

Spark HBase is in technical preview with Spark 1.6.1 and HDP 2.4.2. http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_spark-guide/content/ch_introduction-spark.html

Re: HDP Spark HBase Connector

sunile_manjee — Wed, 18 May 2016 09:07:23 GMT

I found the documentation (not on HDP at this time)

https://github.com/hortonworks/shc

Re: HDP Spark HBase Connector

rhryniewicz — Fri, 10 Jun 2016 03:03:30 GMT

Checkout latest blog on HBase connector: http://hortonworks.com/blog/spark-hbase-dataframe-based-hbase-connector/

Re: HDP Spark HBase Connector

TimothySpann — Sun, 12 Jun 2016 10:46:28 GMT

has anyone written a demo or app on this one?

Re: HDP Spark HBase Connector

Venkateshwarlu_ — Mon, 04 Jul 2016 05:20:51 GMT

Is this technical preview of "Spark HBase Connector" available on latest Sandbox? I believe latest Sandbox is 2.4.0 and technical preview available on 2.4.2? And Github documentation steps to be followed to get this working?

Re: HDP Spark HBase Connector

TimothySpann — Tue, 05 Jul 2016 03:58:39 GMT

check hdp 2.5

Re: HDP Spark HBase Connector

sandyy006 — Tue, 05 Jul 2016 13:58:08 GMT

@Venkat Ankam Here is the latest HDP 2.5 sandbox : https://hortonworks.com/tech-preview-hdp-2-5/

Steps to follow are at https://github.com/hortonworks/tutorials/blob/hdp-2.5/tutorials/hortonworks/spark-hbase-a-dataframe-based-hbase-connector/tutorial.md

Re: HDP Spark HBase Connector

Venkateshwarlu_ — Tue, 05 Jul 2016 20:56:52 GMT

Thanks Sandeep. I will check it out!

Re: HDP Spark HBase Connector

nbalaji-elangov — Tue, 07 Feb 2017 10:33:47 GMT

Is the spark hbase connector GA in HDP 2.5.3 ?

Re: HDP Spark HBase Connector

bikas — Wed, 08 Feb 2017 02:43:20 GMT

Yes. For Spark 1.6 it is GA in HDP 2.5.3. The documentation is available from the Github site for a given SHC release tag. That is the source of truth.

Re: HDP Spark HBase Connector

juan_barbancho_ — Wed, 05 Apr 2017 20:39:40 GMT

I try to use in HDP 2.4.0 and cannot run the samples files. I have a Cluster HDP 2.4

mvn clean package test

Cause: java.net.BindException: Address already in use

at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:433)

....

[INFO] HBase Spark Connector Project Parent POM ........... SUCCESS [ 1.542 s] [INFO] HBase Spark Connector Project Core ................. FAILURE [01:04 min] [INFO] HBase Spark Connector Project Examples ............. SKIPPED

Re: HDP Spark HBase Connector

amirmam — Thu, 06 Jul 2017 14:04:44 GMT

is there a way to use the connector offline? is the connector included in HDP 2.6?

that is without using --repositories http://repo.hortonworks.com/content/groups/public/

i am developing in a closed offline environment and cannot download from the internet the connector...

if not, can anyone provide another way of pyspark-hbase connection?

hi i'm using python pyspark,

is there a way to run spark-submit offline?

that is without using --repositories http://repo.hortonworks.com/content/groups/public/

is the connector included in HDP 2.6?

Re: HDP Spark HBase Connector

schausson — Mon, 23 Oct 2017 21:51:02 GMT

I am in the exact same configuration (no way to reach internet from our cluster...), did you find any other option to make it run ?

Re: HDP Spark HBase Connector

tyu — Tue, 24 Oct 2017 01:21:32 GMT

Please take a look at:

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_spark-component-guide/content/spark-on-hbase.html