You should be able to read HBase Spark connector data via DataFrames in Pyspark, via the sqlContext already today: ~> hbase shell
> create 't', 'c'
> put 't', '1', 'c:a', 'a column data'
> put 't', '1', 'c:b', 'b column data'
> exit
~> export SPARK_CLASSPATH=$(hbase classpath)
~> pyspark
> hTbl = sqlContext.read.format('org.apache.hadoop.hbase.spark')
.option('hbase.table','t')
.option('hbase.columns.mapping', 'KEY_FIELD STRING :key, A STRING c:a, B STRING c:b')
.option('hbase.use.hbase.context', False)
.option('hbase.config.resources', 'file:///etc/hbase/conf/hbase-site.xml')
.load()
> hTbl.show()
+---------+---------+---------+
|KEY_FIELD| A| B|
+---------+---------+---------+
|1|a column data|b column data|
+---------+---------+---------+ There are some limitations as the JIRA notes of course. Which specific missing feature are you looking for, just so we know the scope of request?
... View more
