Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Storm can not be inserted into hive

avatar
New Contributor
Caused by: java.lang.ExceptionInInitializerError
	at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:886) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:642) ~[stormjar.jar:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_112]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_112]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_112]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_112]
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
Caused by: java.lang.RuntimeException: Failed to create local dir /hadoop/hbase/local/jars, DynamicClassLoader failed to init
	at org.apache.hadoop.hbase.util.DynamicClassLoader.initTempDir(DynamicClassLoader.java:108) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.util.DynamicClassLoader.<init>(DynamicClassLoader.java:98) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.protobuf.ProtobufUtil.<clinit>(ProtobufUtil.java:249) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:886) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:642) ~[stormjar.jar:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_112]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_112]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_112]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_112]
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
2017-10-19 17:47:56.468 o.a.z.ClientCnxn Thread-14-hbaseDangerousEvents-executor[11 11]-SendThread(sandbox.hortonworks.com:2181) [INFO] Opening socket connection to server sandbox.hortonworks.com/192.168.1.92:2181. Will not attempt to authenticate using SASL (unknown error)
2017-10-19 17:47:56.469 o.a.z.ClientCnxn Thread-14-hbaseDangerousEvents-executor[11 11]-SendThread(sandbox.hortonworks.com:2181) [INFO] Socket connection established to sandbox.hortonworks.com/192.168.1.92:2181, initiating session
2017-10-19 17:47:56.480 o.a.z.ClientCnxn Thread-14-hbaseDangerousEvents-executor[11 11]-SendThread(sandbox.hortonworks.com:2181) [INFO] Session establishment complete on server sandbox.hortonworks.com/192.168.1.92:2181, sessionid = 0x15f3672e0cd1b5c, negotiated timeout = 30000
2017-10-19 17:47:56.501 o.a.s.util Thread-14-hbaseDangerousEvents-executor[11 11] [ERROR] Async loop died!
java.lang.RuntimeException: HBase bolt preparation failed: java.lang.reflect.InvocationTargetException
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:51) ~[stormjar.jar:?]
	at org.apache.storm.hbase.bolt.AbstractHBaseBolt.prepare(AbstractHBaseBolt.java:75) ~[stormjar.jar:?]
	at org.apache.storm.hbase.bolt.HBaseBolt.prepare(HBaseBolt.java:109) ~[stormjar.jar:?]
	at org.apache.storm.daemon.executor$fn__6573$fn__6586.invoke(executor.clj:798) ~[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
	at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:482) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
	at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_112]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_112]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_112]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_112]
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil
	at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:886) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:642) ~[stormjar.jar:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_112]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_112]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_112]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_112]
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
2017-10-19 17:47:56.503 o.a.s.d.executor Thread-14-hbaseDangerousEvents-executor[11 11] [ERROR] 
java.lang.RuntimeException: HBase bolt preparation failed: java.lang.reflect.InvocationTargetException
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:51) ~[stormjar.jar:?]
	at org.apache.storm.hbase.bolt.AbstractHBaseBolt.prepare(AbstractHBaseBolt.java:75) ~[stormjar.jar:?]
	at org.apache.storm.hbase.bolt.HBaseBolt.prepare(HBaseBolt.java:109) ~[stormjar.jar:?]
	at org.apache.storm.daemon.executor$fn__6573$fn__6586.invoke(executor.clj:798) ~[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
	at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:482) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
	at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] 

Hello

I'm testing the topology of truck-event-processor but i get an error when running the Bolt

1 ACCEPTED SOLUTION

avatar
New Contributor

I found the error, could not create directory /hadoop/hbase/local/jars because they did not have privileges in /hadoop/hbase
Thank you very much anyway

View solution in original post

2 REPLIES 2

avatar
Master Mentor

@Carlos Ramos

The issue seems to be related to CLASSPATH (missing jar in the classpath)

Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil


Can you please verify if your classpath includes the JARs with name like "phoenix-hive.jar" (OR) "storm-starter-topologies-xxx.jar" like following:


Example: (Depending on your HDP version the JAR version might differ)

/usr/hdp/2.6.0.3-8/phoenix/phoenix-hive.jar
/usr/hdp/2.6.0.3-8/storm/contrib/storm-starter/storm-starter-topologies-1.1.0.2.6.0.3-8.jar 

.

avatar
New Contributor

I found the error, could not create directory /hadoop/hbase/local/jars because they did not have privileges in /hadoop/hbase
Thank you very much anyway