Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Storm can not be inserted into hive

avatar
New Member
Caused by: java.lang.ExceptionInInitializerError
	at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:886) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:642) ~[stormjar.jar:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_112]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_112]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_112]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_112]
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
Caused by: java.lang.RuntimeException: Failed to create local dir /hadoop/hbase/local/jars, DynamicClassLoader failed to init
	at org.apache.hadoop.hbase.util.DynamicClassLoader.initTempDir(DynamicClassLoader.java:108) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.util.DynamicClassLoader.<init>(DynamicClassLoader.java:98) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.protobuf.ProtobufUtil.<clinit>(ProtobufUtil.java:249) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:886) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:642) ~[stormjar.jar:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_112]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_112]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_112]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_112]
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
2017-10-19 17:47:56.468 o.a.z.ClientCnxn Thread-14-hbaseDangerousEvents-executor[11 11]-SendThread(sandbox.hortonworks.com:2181) [INFO] Opening socket connection to server sandbox.hortonworks.com/192.168.1.92:2181. Will not attempt to authenticate using SASL (unknown error)
2017-10-19 17:47:56.469 o.a.z.ClientCnxn Thread-14-hbaseDangerousEvents-executor[11 11]-SendThread(sandbox.hortonworks.com:2181) [INFO] Socket connection established to sandbox.hortonworks.com/192.168.1.92:2181, initiating session
2017-10-19 17:47:56.480 o.a.z.ClientCnxn Thread-14-hbaseDangerousEvents-executor[11 11]-SendThread(sandbox.hortonworks.com:2181) [INFO] Session establishment complete on server sandbox.hortonworks.com/192.168.1.92:2181, sessionid = 0x15f3672e0cd1b5c, negotiated timeout = 30000
2017-10-19 17:47:56.501 o.a.s.util Thread-14-hbaseDangerousEvents-executor[11 11] [ERROR] Async loop died!
java.lang.RuntimeException: HBase bolt preparation failed: java.lang.reflect.InvocationTargetException
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:51) ~[stormjar.jar:?]
	at org.apache.storm.hbase.bolt.AbstractHBaseBolt.prepare(AbstractHBaseBolt.java:75) ~[stormjar.jar:?]
	at org.apache.storm.hbase.bolt.HBaseBolt.prepare(HBaseBolt.java:109) ~[stormjar.jar:?]
	at org.apache.storm.daemon.executor$fn__6573$fn__6586.invoke(executor.clj:798) ~[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
	at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:482) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
	at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_112]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_112]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_112]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_112]
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil
	at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:886) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:642) ~[stormjar.jar:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_112]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_112]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_112]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_112]
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
2017-10-19 17:47:56.503 o.a.s.d.executor Thread-14-hbaseDangerousEvents-executor[11 11] [ERROR] 
java.lang.RuntimeException: HBase bolt preparation failed: java.lang.reflect.InvocationTargetException
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:51) ~[stormjar.jar:?]
	at org.apache.storm.hbase.bolt.AbstractHBaseBolt.prepare(AbstractHBaseBolt.java:75) ~[stormjar.jar:?]
	at org.apache.storm.hbase.bolt.HBaseBolt.prepare(HBaseBolt.java:109) ~[stormjar.jar:?]
	at org.apache.storm.daemon.executor$fn__6573$fn__6586.invoke(executor.clj:798) ~[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
	at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:482) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
	at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] 

Hello

I'm testing the topology of truck-event-processor but i get an error when running the Bolt

1 ACCEPTED SOLUTION

avatar
New Member

I found the error, could not create directory /hadoop/hbase/local/jars because they did not have privileges in /hadoop/hbase
Thank you very much anyway

View solution in original post

2 REPLIES 2

avatar
Master Mentor

@Carlos Ramos

The issue seems to be related to CLASSPATH (missing jar in the classpath)

Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil


Can you please verify if your classpath includes the JARs with name like "phoenix-hive.jar" (OR) "storm-starter-topologies-xxx.jar" like following:


Example: (Depending on your HDP version the JAR version might differ)

/usr/hdp/2.6.0.3-8/phoenix/phoenix-hive.jar
/usr/hdp/2.6.0.3-8/storm/contrib/storm-starter/storm-starter-topologies-1.1.0.2.6.0.3-8.jar 

.

avatar
New Member

I found the error, could not create directory /hadoop/hbase/local/jars because they did not have privileges in /hadoop/hbase
Thank you very much anyway