Archives of Support Questions (Read Only)

cui_lin · ‎12-15-2015

What's the best practice to get data from hbase and form dataframe for Python/R? If we want to use our Panda/R libraries, how to get data from hbase and form dataframe automatically?

vshukla · ‎01-08-2016

We have an experimental Spark HBase connector, https://github.com/zhzhan/shc

With the following features

First class support for DataFrame API
JSON based catalog with rich data type support
Performant, scalable, enterprise-ready
Partition Pruning
Predicate Pushdown
Scan optimizations
Data Locality
Composite Rowkey
Leverage existing work in the HBase community

Please take look at the README of the above project.

Also see example https://github.com/zhzhan/shc/blob/master/src/main/scala/org/apache/spark/sql/execution/datasources/...

View solution in original post

sai_geetha · ‎12-28-2017

@Artem Ervits, Is there any progress on the Spark on HBase by Hortonworks. We are using the HDP platform but I am not able to easily conclude from the internet that confirms there is progress beyond the above discussion in 2016.

aervits · ‎01-25-2018

@Sai Geetha M N please read our latest docs https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_spark-component-guide/content/ch_introdu...

and https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_spark-component-guide/content/ch08s05.ht... it's been out for a while now.