Using Spark Hbase in HDP 2.3.2

[#1]

I am using HDP 2.3.2 with spark 1.4.1. As per below link Spark hbase connector works with HDP 2.4.2 onwards. Can someone help with me how can i read hbase in spark using HDP 2.3.2

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_spark-guide/content/ch_introduction-spark...

[#2 · Accepted solution]

Hi @pooja khandelwal As mentioned in the same doc link, spark 1.6.1 comes with spark hbase connector which under Technical preview(TP), moreover spark 1.6.1 is only certified on HDP 2.4.2. On the other side HDP 2.3.2 has spark 1.4.1 which doesn't support spark Hbase connector in HDP.

[#3]

I understand that. I am asking is there any other way I can read hbase inside spark?

[#4]

The only recommended way is to upgrade the cluster to HDP2.4.2. You can also install spark 1.6.1 manually on HDP 2.3.2 but we don't recommend that.

Thanks.

[#5]

I'm reading from Hbase in a spark Hdp 2.3.4.0-3485 (and I've done reading with lower Hdp releases).

Read hbase through a RDD:

Configuration hbaseConfiguration = HBaseConfiguration.create();
hbaseConfiguration.set(TableInputFormat.INPUT_TABLE, "sometable");
JavaPairRDD<ImmutableBytesWritable, Result> hbaseRdd = sc.newAPIHadoopRDD(hbaseConfiguration, TableInputFormat.class, ImmutableBytesWritable.class, Result.class);

Or am I missing something?