Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive HBase snapshot query results in exception

Solved Go to solution
Highlighted

Hive HBase snapshot query results in exception

Explorer

Hi, we are using CDH5.4.4 and I am trying to use Hive to query HBase snapshots. But when I issue a query I am getting the following exception in the hive CLI

 

Failed with exception java.io.IOException:java.io.IOException: java.lang.NoClassDefFoundError: com/yammer/metrics/core/MetricsRegistry

 

Here are the steps I took

 

1. Created an HBase snapshot 'snapshot_20150827'

2. In Hive CLI executed the following

a. Create external table

     CREATE EXTERNAL TABLE e(id STRING,.....) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
     WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,....")

     TBLPROPERTIES ("hbase.table.name" = "e",  "hbase.table.default.storage.type" = "binary")

b. SET hive.hbase.snapshot.name=snapshot_20150827;

c.  Execute the query

     select * from e where id = '...';

 

and this gives me the exception noted above, please help.

 

Nitin

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Hive HBase snapshot query results in exception

Master Guru
Looks like the jar is also required on the front-end. In addition to the
ADD JAR within the prompt, please also launch the CLI instead in this way:

~> export HADOOP_CLASSPATH=$(hbase classpath)
~> hive

View solution in original post

7 REPLIES 7
Highlighted

Re: Hive HBase snapshot query results in exception

Master Guru
The snapshot read path uses a few more jars than the default table read path code does, and the error suggests that at least one such extra jar is not on the default set of aux-jars pre-added for Hive-HBase integration in CDH.

You will need to do an "ADD JAR /opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar;" to get this required class on the Hive CLI classpath.
Highlighted

Re: Hive HBase snapshot query results in exception

Explorer

Thanks for the reply Harsh. But that did not help. I tried ADD JAR but still getting the same error. Here is what I see from hive shell

 

hive> ADD JAR /opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar;
Added [/opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar] to class path
Added resources: [/opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar]
hive> SET hive.hbase.snapshot.name=snapshot_20150920
    > ;
hive> select * from e where id = '...';
OK
Failed with exception java.io.IOException:java.io.IOException: java.lang.NoClassDefFoundError: com/yammer/metrics/core/MetricsRegistry
Time taken: 36.085 seconds
hive>

 

I should add that the jar file does exist at the location in ADD JAR

 

$ ls -ltr /opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar
lrwxrwxrwx 1 root root 36 Aug 13 14:40 /opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar -> ../../../jars/metrics-core-2.2.0.jar

Highlighted

Re: Hive HBase snapshot query results in exception

Master Guru
Looks like the jar is also required on the front-end. In addition to the
ADD JAR within the prompt, please also launch the CLI instead in this way:

~> export HADOOP_CLASSPATH=$(hbase classpath)
~> hive

View solution in original post

Highlighted

Re: Hive HBase snapshot query results in exception

Explorer

No go yet Harsh, same error again. Here is what I did.

 

$ export HADOOP_CLASSPATH=$(hbase classpath)
$ sudo -u hdfs hive

Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/jars/hive-common-1.1.0-cdh5.4.4.jar!/hive-log4j.properties
WARNING: Hive CLI is deprecated and migration to Beeline is recommended.
hive> ADD JAR /opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar;
Added [/opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar] to class path
Added resources: [/opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar]
hive> SET hive.hbase.snapshot.name=snapshot_20150920;
hive> select * from e where id = '...';
OK
Failed with exception java.io.IOException:java.io.IOException: java.lang.NoClassDefFoundError: com/yammer/metrics/core/MetricsRegistry
Time taken: 28.503 seconds
hive>

 

I did verify that $(hbase classpath) has the metrics-core-2.2.0.jar.

Highlighted

Re: Hive HBase snapshot query results in exception

Master Guru
The use of sudo will not pass forward your local environment. Login and try
instead.

Highlighted

Re: Hive HBase snapshot query results in exception

Explorer

Thanks Harsh, that was it. It works!

Highlighted

Re: Hive HBase snapshot query results in exception

New Contributor

We too have the similar issue while reading from the hbase snapshot . Here is our senario
Created a table in HBASE (create 'TEST_HBASE_TABLE','cf')
Created a hive external hbase table with name TEST_EXTERNAL_HBASE_TABLE
created snapshot for (snapshot 'TEST_HBASE_TABLE','TEST_SNAPSHOT')
set the hive hbase property (set hive.hbase.snapshot.name=TEST_SNAPSHOT)
execute the query ( select * from TEST_EXTERNAL_HBASE_TABLE )

got the below exception . Need some help in fixing this issue

2015-09-28 16:10:29,025 WARN org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hbase_user (auth:SIMPLE) cause:org.apache.hive.service.cli.Hivetion: java.io.IOException: java.io.IOException: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.io.hfile.CacheStats

org.apache.hive.service.cli.HiveSQLException: java.io.IOException: java.io.IOException: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbasle.CacheStats
at org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:329)
at org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:250)
at org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:699)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
at com.sun.proxy.$Proxy17.fetchResults(Unknown Source)
at org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:451)
at org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:676)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1553)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1538)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: java.io.IOException: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.io.hfile.CacheStats
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:507)
at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:414)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:138)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1657)
at org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:324)
... 24 more
Caused by: java.io.IOException: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.io.hfile.CacheStats
at org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionStores(HRegion.java:951)
at org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:841)
at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:814)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5828)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5794)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5765)
at org.apache.hadoop.hbase.client.ClientSideRegionScanner.<init>(ClientSideRegionScanner.java:57)
at org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormatImpl$RecordReader.initialize(TableSnapshotInputFormatImpl.java:190)
at org.apache.hadoop.hbase.mapred.TableSnapshotInputFormat$TableSnapshotRecordReader.<init>(TableSnapshotInputFormat.java:96)
at org.apache.hadoop.hbase.mapred.TableSnapshotInputFormat.getRecordReader(TableSnapshotInputFormat.java:150)
at org.apache.hadoop.hive.hbase.HiveHBaseTableSnapshotInputFormat.getRecordReader(HiveHBaseTableSnapshotInputFormat.java:74)

Don't have an account?
Coming from Hortonworks? Activate your account here