Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Spark sqlContext - unable to access hbase table mapped in hive as external table

avatar
Contributor

Env: Sandbox in VMware.

I am trying to access Hbase table mapped in Hive and failing with below error. Am i missing in the configuration part.

scala> val res = sqlContext.sql("select * from hbase_user_act")
16/08/31 20:57:06 INFO ParseDriver: Parsing command: select * from hbase_user_act
16/08/31 20:57:07 INFO ParseDriver: Parse Completed
16/08/31 20:57:07 ERROR log: error in initSerDe: java.lang.ClassNotFoundException Class org.apache.hadoop.hive.hbase.HBaseSerDe not found
java.lang.ClassNotFoundException: Class org.apache.hadoop.hive.hbase.HBaseSerDe not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
...
...

java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Error in loading storage handler.org.apache.hadoop.hive.hbase.HBaseStorageHandler
...
...
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.hbase.HBaseStorageHandler

8 REPLIES 8

avatar
Contributor

The issue is happening in both spark-shell and pyspark shell.

avatar
Contributor

@jgeorge : Could you please with this one.

avatar
Super Guru

avatar
Contributor

@mqureshi Even with HiveContext, the error still persists.

avatar
Super Guru

@Madhu B

There are several ways to skin this cat but they would require some classpath tricks. Before going there, could you create in Hive a view of that table, e.g. create view hbase_user_act_view as select * from hbase_user_act; and test with that? Use HiveContext, please. Let me know.

If any of the responses in this thread addressed your issue, don't forget to vote and accept best answer.

avatar
Contributor

@Constantin Stanca I am using Zeppelin with Spark Interpreter to access the hive-hbase table. I get the following error while running the query:

java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Error in loading storage handler.org.apache.hadoop.hive.hbase.HBaseStorageHandler

I have added

even the required jars in classpath for Spark interpreter but still the issue persists:

38548-zeppelin-classpath.png

Could you please let me know what could be the problem?

avatar
New Contributor

@mb

I am facing the same issue, could you please advice how to work around or troubleshoot this problem ?

Thanks,

Nam

,

HI I'm facing the same issue. Could you please provide some hint or solutions ?

avatar
New Contributor

@mb

I am facing the same issue, could you please advice how to work around or troubleshoot this problem ?

Thanks,

Nam