Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hive HBase snapshot query results in exception

avatar
Explorer

Hi, we are using CDH5.4.4 and I am trying to use Hive to query HBase snapshots. But when I issue a query I am getting the following exception in the hive CLI

 

Failed with exception java.io.IOException:java.io.IOException: java.lang.NoClassDefFoundError: com/yammer/metrics/core/MetricsRegistry

 

Here are the steps I took

 

1. Created an HBase snapshot 'snapshot_20150827'

2. In Hive CLI executed the following

a. Create external table

     CREATE EXTERNAL TABLE e(id STRING,.....) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
     WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,....")

     TBLPROPERTIES ("hbase.table.name" = "e",  "hbase.table.default.storage.type" = "binary")

b. SET hive.hbase.snapshot.name=snapshot_20150827;

c.  Execute the query

     select * from e where id = '...';

 

and this gives me the exception noted above, please help.

 

Nitin

1 ACCEPTED SOLUTION

avatar
Mentor
Looks like the jar is also required on the front-end. In addition to the
ADD JAR within the prompt, please also launch the CLI instead in this way:

~> export HADOOP_CLASSPATH=$(hbase classpath)
~> hive

View solution in original post

7 REPLIES 7

avatar
Mentor
The snapshot read path uses a few more jars than the default table read path code does, and the error suggests that at least one such extra jar is not on the default set of aux-jars pre-added for Hive-HBase integration in CDH.

You will need to do an "ADD JAR /opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar;" to get this required class on the Hive CLI classpath.

avatar
Explorer

Thanks for the reply Harsh. But that did not help. I tried ADD JAR but still getting the same error. Here is what I see from hive shell

 

hive> ADD JAR /opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar;
Added [/opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar] to class path
Added resources: [/opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar]
hive> SET hive.hbase.snapshot.name=snapshot_20150920
    > ;
hive> select * from e where id = '...';
OK
Failed with exception java.io.IOException:java.io.IOException: java.lang.NoClassDefFoundError: com/yammer/metrics/core/MetricsRegistry
Time taken: 36.085 seconds
hive>

 

I should add that the jar file does exist at the location in ADD JAR

 

$ ls -ltr /opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar
lrwxrwxrwx 1 root root 36 Aug 13 14:40 /opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar -> ../../../jars/metrics-core-2.2.0.jar

avatar
Mentor
Looks like the jar is also required on the front-end. In addition to the
ADD JAR within the prompt, please also launch the CLI instead in this way:

~> export HADOOP_CLASSPATH=$(hbase classpath)
~> hive

avatar
Explorer

No go yet Harsh, same error again. Here is what I did.

 

$ export HADOOP_CLASSPATH=$(hbase classpath)
$ sudo -u hdfs hive

Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/jars/hive-common-1.1.0-cdh5.4.4.jar!/hive-log4j.properties
WARNING: Hive CLI is deprecated and migration to Beeline is recommended.
hive> ADD JAR /opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar;
Added [/opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar] to class path
Added resources: [/opt/cloudera/parcels/CDH/lib/hbase/lib/metrics-core-2.2.0.jar]
hive> SET hive.hbase.snapshot.name=snapshot_20150920;
hive> select * from e where id = '...';
OK
Failed with exception java.io.IOException:java.io.IOException: java.lang.NoClassDefFoundError: com/yammer/metrics/core/MetricsRegistry
Time taken: 28.503 seconds
hive>

 

I did verify that $(hbase classpath) has the metrics-core-2.2.0.jar.

avatar
Mentor
The use of sudo will not pass forward your local environment. Login and try
instead.

avatar
Explorer

Thanks Harsh, that was it. It works!

avatar
New Contributor

We too have the similar issue while reading from the hbase snapshot . Here is our senario
Created a table in HBASE (create 'TEST_HBASE_TABLE','cf')
Created a hive external hbase table with name TEST_EXTERNAL_HBASE_TABLE
created snapshot for (snapshot 'TEST_HBASE_TABLE','TEST_SNAPSHOT')
set the hive hbase property (set hive.hbase.snapshot.name=TEST_SNAPSHOT)
execute the query ( select * from TEST_EXTERNAL_HBASE_TABLE )

got the below exception . Need some help in fixing this issue

2015-09-28 16:10:29,025 WARN org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hbase_user (auth:SIMPLE) cause:org.apache.hive.service.cli.Hivetion: java.io.IOException: java.io.IOException: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.io.hfile.CacheStats

org.apache.hive.service.cli.HiveSQLException: java.io.IOException: java.io.IOException: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbasle.CacheStats
at org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:329)
at org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:250)
at org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:699)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
at com.sun.proxy.$Proxy17.fetchResults(Unknown Source)
at org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:451)
at org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:676)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1553)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1538)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: java.io.IOException: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.io.hfile.CacheStats
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:507)
at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:414)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:138)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1657)
at org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:324)
... 24 more
Caused by: java.io.IOException: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.io.hfile.CacheStats
at org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionStores(HRegion.java:951)
at org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:841)
at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:814)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5828)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5794)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5765)
at org.apache.hadoop.hbase.client.ClientSideRegionScanner.<init>(ClientSideRegionScanner.java:57)
at org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormatImpl$RecordReader.initialize(TableSnapshotInputFormatImpl.java:190)
at org.apache.hadoop.hbase.mapred.TableSnapshotInputFormat$TableSnapshotRecordReader.<init>(TableSnapshotInputFormat.java:96)
at org.apache.hadoop.hbase.mapred.TableSnapshotInputFormat.getRecordReader(TableSnapshotInputFormat.java:150)
at org.apache.hadoop.hive.hbase.HiveHBaseTableSnapshotInputFormat.getRecordReader(HiveHBaseTableSnapshotInputFormat.java:74)