Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark sqlContext - unable to access hbase table mapped in hive as external table

Spark sqlContext - unable to access hbase table mapped in hive as external table

New Contributor

Env: Sandbox in VMware.

I am trying to access Hbase table mapped in Hive and failing with below error. Am i missing in the configuration part.

scala> val res = sqlContext.sql("select * from hbase_user_act")
16/08/31 20:57:06 INFO ParseDriver: Parsing command: select * from hbase_user_act
16/08/31 20:57:07 INFO ParseDriver: Parse Completed
16/08/31 20:57:07 ERROR log: error in initSerDe: java.lang.ClassNotFoundException Class org.apache.hadoop.hive.hbase.HBaseSerDe not found
java.lang.ClassNotFoundException: Class org.apache.hadoop.hive.hbase.HBaseSerDe not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
...
...

java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Error in loading storage handler.org.apache.hadoop.hive.hbase.HBaseStorageHandler
...
...
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.hbase.HBaseStorageHandler

8 REPLIES 8

Re: Spark sqlContext - unable to access hbase table mapped in hive as external table

New Contributor

The issue is happening in both spark-shell and pyspark shell.

Re: Spark sqlContext - unable to access hbase table mapped in hive as external table

New Contributor

@jgeorge : Could you please with this one.

Re: Spark sqlContext - unable to access hbase table mapped in hive as external table

Super Guru

Re: Spark sqlContext - unable to access hbase table mapped in hive as external table

New Contributor

@mqureshi Even with HiveContext, the error still persists.

Re: Spark sqlContext - unable to access hbase table mapped in hive as external table

@Madhu B

There are several ways to skin this cat but they would require some classpath tricks. Before going there, could you create in Hive a view of that table, e.g. create view hbase_user_act_view as select * from hbase_user_act; and test with that? Use HiveContext, please. Let me know.

If any of the responses in this thread addressed your issue, don't forget to vote and accept best answer.

Re: Spark sqlContext - unable to access hbase table mapped in hive as external table

New Contributor

@Constantin Stanca I am using Zeppelin with Spark Interpreter to access the hive-hbase table. I get the following error while running the query:

java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Error in loading storage handler.org.apache.hadoop.hive.hbase.HBaseStorageHandler

I have added

even the required jars in classpath for Spark interpreter but still the issue persists:

38548-zeppelin-classpath.png

Could you please let me know what could be the problem?

Re: Spark sqlContext - unable to access hbase table mapped in hive as external table

New Contributor

@mb

I am facing the same issue, could you please advice how to work around or troubleshoot this problem ?

Thanks,

Nam

,

HI I'm facing the same issue. Could you please provide some hint or solutions ?

Re: Spark sqlContext - unable to access hbase table mapped in hive as external table

New Contributor

@mb

I am facing the same issue, could you please advice how to work around or troubleshoot this problem ?

Thanks,

Nam

Don't have an account?
Coming from Hortonworks? Activate your account here