Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Storm can not be inserted into hive

Solved Go to solution
Highlighted

Storm can not be inserted into hive

New Contributor
Caused by: java.lang.ExceptionInInitializerError
	at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:886) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:642) ~[stormjar.jar:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_112]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_112]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_112]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_112]
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
Caused by: java.lang.RuntimeException: Failed to create local dir /hadoop/hbase/local/jars, DynamicClassLoader failed to init
	at org.apache.hadoop.hbase.util.DynamicClassLoader.initTempDir(DynamicClassLoader.java:108) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.util.DynamicClassLoader.<init>(DynamicClassLoader.java:98) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.protobuf.ProtobufUtil.<clinit>(ProtobufUtil.java:249) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:886) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:642) ~[stormjar.jar:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_112]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_112]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_112]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_112]
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
2017-10-19 17:47:56.468 o.a.z.ClientCnxn Thread-14-hbaseDangerousEvents-executor[11 11]-SendThread(sandbox.hortonworks.com:2181) [INFO] Opening socket connection to server sandbox.hortonworks.com/192.168.1.92:2181. Will not attempt to authenticate using SASL (unknown error)
2017-10-19 17:47:56.469 o.a.z.ClientCnxn Thread-14-hbaseDangerousEvents-executor[11 11]-SendThread(sandbox.hortonworks.com:2181) [INFO] Socket connection established to sandbox.hortonworks.com/192.168.1.92:2181, initiating session
2017-10-19 17:47:56.480 o.a.z.ClientCnxn Thread-14-hbaseDangerousEvents-executor[11 11]-SendThread(sandbox.hortonworks.com:2181) [INFO] Session establishment complete on server sandbox.hortonworks.com/192.168.1.92:2181, sessionid = 0x15f3672e0cd1b5c, negotiated timeout = 30000
2017-10-19 17:47:56.501 o.a.s.util Thread-14-hbaseDangerousEvents-executor[11 11] [ERROR] Async loop died!
java.lang.RuntimeException: HBase bolt preparation failed: java.lang.reflect.InvocationTargetException
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:51) ~[stormjar.jar:?]
	at org.apache.storm.hbase.bolt.AbstractHBaseBolt.prepare(AbstractHBaseBolt.java:75) ~[stormjar.jar:?]
	at org.apache.storm.hbase.bolt.HBaseBolt.prepare(HBaseBolt.java:109) ~[stormjar.jar:?]
	at org.apache.storm.daemon.executor$fn__6573$fn__6586.invoke(executor.clj:798) ~[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
	at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:482) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
	at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_112]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_112]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_112]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_112]
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil
	at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:886) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:642) ~[stormjar.jar:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_112]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_112]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_112]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_112]
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
2017-10-19 17:47:56.503 o.a.s.d.executor Thread-14-hbaseDangerousEvents-executor[11 11] [ERROR] 
java.lang.RuntimeException: HBase bolt preparation failed: java.lang.reflect.InvocationTargetException
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:51) ~[stormjar.jar:?]
	at org.apache.storm.hbase.bolt.AbstractHBaseBolt.prepare(AbstractHBaseBolt.java:75) ~[stormjar.jar:?]
	at org.apache.storm.hbase.bolt.HBaseBolt.prepare(HBaseBolt.java:109) ~[stormjar.jar:?]
	at org.apache.storm.daemon.executor$fn__6573$fn__6586.invoke(executor.clj:798) ~[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
	at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:482) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
	at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] 

Hello

I'm testing the topology of truck-event-processor but i get an error when running the Bolt

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Storm can not be inserted into hive

New Contributor

I found the error, could not create directory /hadoop/hbase/local/jars because they did not have privileges in /hadoop/hbase
Thank you very much anyway

View solution in original post

2 REPLIES 2

Re: Storm can not be inserted into hive

Super Mentor

@Carlos Ramos

The issue seems to be related to CLASSPATH (missing jar in the classpath)

Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil


Can you please verify if your classpath includes the JARs with name like "phoenix-hive.jar" (OR) "storm-starter-topologies-xxx.jar" like following:


Example: (Depending on your HDP version the JAR version might differ)

/usr/hdp/2.6.0.3-8/phoenix/phoenix-hive.jar
/usr/hdp/2.6.0.3-8/storm/contrib/storm-starter/storm-starter-topologies-1.1.0.2.6.0.3-8.jar 

.

Highlighted

Re: Storm can not be inserted into hive

New Contributor

I found the error, could not create directory /hadoop/hbase/local/jars because they did not have privileges in /hadoop/hbase
Thank you very much anyway

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here