Storm can not be inserted into hive

yolps19 — Fri, 20 Oct 2017 20:46:32 GMT

Caused by: java.lang.ExceptionInInitializerError
	at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:886) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:642) ~[stormjar.jar:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_112]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_112]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_112]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_112]
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
Caused by: java.lang.RuntimeException: Failed to create local dir /hadoop/hbase/local/jars, DynamicClassLoader failed to init
	at org.apache.hadoop.hbase.util.DynamicClassLoader.initTempDir(DynamicClassLoader.java:108) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.util.DynamicClassLoader.<init>(DynamicClassLoader.java:98) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.protobuf.ProtobufUtil.<clinit>(ProtobufUtil.java:249) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:886) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:642) ~[stormjar.jar:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_112]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_112]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_112]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_112]
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
2017-10-19 17:47:56.468 o.a.z.ClientCnxn Thread-14-hbaseDangerousEvents-executor[11 11]-SendThread(sandbox.hortonworks.com:2181) [INFO] Opening socket connection to server sandbox.hortonworks.com/192.168.1.92:2181. Will not attempt to authenticate using SASL (unknown error)
2017-10-19 17:47:56.469 o.a.z.ClientCnxn Thread-14-hbaseDangerousEvents-executor[11 11]-SendThread(sandbox.hortonworks.com:2181) [INFO] Socket connection established to sandbox.hortonworks.com/192.168.1.92:2181, initiating session
2017-10-19 17:47:56.480 o.a.z.ClientCnxn Thread-14-hbaseDangerousEvents-executor[11 11]-SendThread(sandbox.hortonworks.com:2181) [INFO] Session establishment complete on server sandbox.hortonworks.com/192.168.1.92:2181, sessionid = 0x15f3672e0cd1b5c, negotiated timeout = 30000
2017-10-19 17:47:56.501 o.a.s.util Thread-14-hbaseDangerousEvents-executor[11 11] [ERROR] Async loop died!
java.lang.RuntimeException: HBase bolt preparation failed: java.lang.reflect.InvocationTargetException
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:51) ~[stormjar.jar:?]
	at org.apache.storm.hbase.bolt.AbstractHBaseBolt.prepare(AbstractHBaseBolt.java:75) ~[stormjar.jar:?]
	at org.apache.storm.hbase.bolt.HBaseBolt.prepare(HBaseBolt.java:109) ~[stormjar.jar:?]
	at org.apache.storm.daemon.executor$fn__6573$fn__6586.invoke(executor.clj:798) ~[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
	at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:482) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
	at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_112]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_112]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_112]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_112]
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil
	at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:886) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:642) ~[stormjar.jar:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_112]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_112]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_112]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_112]
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:420) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:298) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:184) ~[stormjar.jar:?]
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:47) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient$1.run(HBaseClient.java:44) ~[stormjar.jar:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) ~[stormjar.jar:?]
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:44) ~[stormjar.jar:?]
	... 6 more
2017-10-19 17:47:56.503 o.a.s.d.executor Thread-14-hbaseDangerousEvents-executor[11 11] [ERROR] 
java.lang.RuntimeException: HBase bolt preparation failed: java.lang.reflect.InvocationTargetException
	at org.apache.storm.hbase.common.HBaseClient.<init>(HBaseClient.java:51) ~[stormjar.jar:?]
	at org.apache.storm.hbase.bolt.AbstractHBaseBolt.prepare(AbstractHBaseBolt.java:75) ~[stormjar.jar:?]
	at org.apache.storm.hbase.bolt.HBaseBolt.prepare(HBaseBolt.java:109) ~[stormjar.jar:?]
	at org.apache.storm.daemon.executor$fn__6573$fn__6586.invoke(executor.clj:798) ~[storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
	at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:482) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
	at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]

Hello

I'm testing the topology of truck-event-processor but i get an error when running the Bolt

Re: Storm can not be inserted into hive

jsensharma — Sat, 21 Oct 2017 00:42:33 GMT

@Carlos Ramos

The issue seems to be related to CLASSPATH (missing jar in the classpath)

Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil

Can you please verify if your classpath includes the JARs with name like "phoenix-hive.jar" (OR) "storm-starter-topologies-xxx.jar" like following:

Example: (Depending on your HDP version the JAR version might differ)

/usr/hdp/2.6.0.3-8/phoenix/phoenix-hive.jar
/usr/hdp/2.6.0.3-8/storm/contrib/storm-starter/storm-starter-topologies-1.1.0.2.6.0.3-8.jar

Re: Storm can not be inserted into hive

yolps19 — Sat, 21 Oct 2017 00:51:41 GMT

I found the error, could not create directory /hadoop/hbase/local/jars because they did not have privileges in /hadoop/hbase
Thank you very much anyway

question Re: Storm can not be inserted into hive in Archives of Support Questions (Read Only)

Storm can not be inserted into hive

Re: Storm can not be inserted into hive

Re: Storm can not be inserted into hive