Hbase Connectivity Fails

[#1]

Getting following error while connecting to Hbase.

HConnection connection = HConnectionManager.createConnection(conf);

Error in Log file:

2015-10-07 20:19:33 o.a.z.ClientCnxn [INFO] Session establishment complete on server localhost.localdomain/127.0.0.1:2181, sessionid = 0x15043a5de090013, negotiated timeout = 4000

2015-10-07 20:19:33 STDIO [ERROR] java.io.IOException: java.lang.reflect.InvocationTargetException

2015-10-07 20:19:33 STDIO [ERROR] at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:426)

2015-10-07 20:19:33 STDIO [ERROR] at org.apache.hadoop.hbase.client.ConnectionManager.createConnectionInternal(ConnectionManager.java:319)

2015-10-07 20:19:33 STDIO [ERROR] at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:292)

2015-10-07 20:19:33 STDIO [ERROR] at com.opensoc.enrichment.adapters.write.dhcp.DynamicLookupWriterBolt.initializeAdapter(DynamicLookupWriterBolt.java:64)

2015-10-07 20:19:33 STDIO [ERROR] at com.opensoc.enrichment.adapters.write.dhcp.DynamicLookupWriterBolt.execute(DynamicLookupWriterBolt.java:115)

2015-10-07 20:19:33 STDIO [ERROR] at backtype.storm.topology.BasicBoltExecutor.execute(BasicBoltExecutor.java:50)

2015-10-07 20:19:33 STDIO [ERROR] at backtype.storm.daemon.executor$fn__5697$tuple_action_fn__5699.invoke(executor.clj:659)

2015-10-07 20:19:33 STDIO [ERROR] at backtype.storm.daemon.executor$mk_task_receiver$fn__5620.invoke(executor.clj:415)

2015-10-07 20:19:33 STDIO [ERROR] at backtype.storm.disruptor$clojure_handler$reify__1741.onEvent(disruptor.clj:58)

2015-10-07 20:19:33 STDIO [ERROR] at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)

2015-10-07 20:19:33 STDIO [ERROR] at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)

2015-10-07 20:19:33 STDIO [ERROR] at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)

2015-10-07 20:19:33 STDIO [ERROR] at backtype.storm.daemon.executor$fn__5697$fn__5710$fn__5761.invoke(executor.clj:794)

2015-10-07 20:19:33 STDIO [ERROR] at backtype.storm.util$async_loop$fn__452.invoke(util.clj:465)

2015-10-07 20:19:33 STDIO [ERROR] at clojure.lang.AFn.run(AFn.java:24)

2015-10-07 20:19:33 STDIO [ERROR] at java.lang.Thread.run(Thread.java:745)

2015-10-07 20:19:33 STDIO [ERROR] Caused by: java.lang.reflect.InvocationTargetException

2015-10-07 20:19:33 STDIO [ERROR] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

2015-10-07 20:19:33 STDIO [ERROR] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

2015-10-07 20:19:33 STDIO [ERROR] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

2015-10-07 20:19:33 STDIO [ERROR] at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

2015-10-07 20:19:33 STDIO [ERROR] at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:424)

2015-10-07 20:19:33 STDIO [ERROR] ... 15 more

2015-10-07 20:19:33 STDIO [ERROR] Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil

2015-10-07 20:19:33 STDIO [ERROR] at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)

2015-10-07 20:19:33 STDIO [ERROR] at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)

2015-10-07 20:19:33 STDIO [ERROR] at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:106)

2015-10-07 20:19:33 STDIO [ERROR] at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:858)

2015-10-07 20:19:33 STDIO [ERROR] at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:662)

2015-10-07 20:19:33 STDIO [ERROR] ... 20 more

[#2 · Accepted solution]

Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil

Looks like you don't have the correct JARs on the classpath. Hard to say why you might be missing that class without any product version information or the classpath/command for Storm.

[#3]

Hi,

Is the issue resolved. I am also facing the same issue when I tried to connect HBase via Java application.

Am using HBase 1.1.2 jar. Kept core-site, hdfs-site, hbase-site.xml files in application along with jars.

Please let me know if your resolved.

thanks,

Swarupa

[#4]

I have the same issue as well. From the stack trace, I see the exception is coming from the Jetty server immediately after establishing a connection with Zookeeper.

Context: I am running HBase 1.1.2 & Phoenix 4.4.0 on HDP 2.4 (Ambari version is 2.2.1.0)

Following is the stack trace I captured from launching sqlline-thin.py

karthick@phoenix-client-staging: 1 $ grep ProtobufUtil strace.17028
recvfrom(16, "HTTP/1.1 500 Server Error\r\nDate: Fri, 18 Nov 2016 21:19:02 GMT\r\nContent-Type: application/octet-stream;charset=utf-8\r\nContent-Length: 4983\r\nServer: Jetty(9.2.z-SNAPSHOT)\r\n\r\n\n8org.apache.calcite.avatica.proto.Responses$ErrorResponse\22\272&\n\222#java.lang.RuntimeException: java.sql.SQLException: ERROR 103 (08004): Unable to establish connection.\n\tat org.apache.calcite.avatica.jdbc.JdbcMeta.openConnection(JdbcMeta.java:579)\n\tat org.apache.calcite.avatica.remote.LocalService.apply(LocalService.java:263)\n\tat org.apache.calcite.avatica.remote.Service$OpenConnectionRequest.accept(Service.java:2026)\n\tat org.apache.calcite.avatica.remote.Service$OpenConnectionRequest.accept(Service.java:2009)\n\tat org.apache.calcite.avatica.remote.AbstractHandler.apply(AbstractHandler.java:102)\n\tat org.apache.calcite.avatica.remote.ProtobufHandler.apply(ProtobufHandler.java:38)\n\tat org.apache.calcite.avatica.server.AvaticaProtobufHandler.handle(AvaticaProtobufHandler.java:68)\n\tat org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:497)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:245)\n\tat org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)\n\tat java.lang.Thread.run(Thread.java:745)\nCaused by: java.sql.SQLException: ERROR 103 (08004): Unable to establish connection.\n\tat org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:395)\n\tat org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:287)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl.access$300(ConnectionQueryServicesImpl.java:170)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:1840)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:1819)\n\tat org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:77)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:1819)\n\tat org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:180)\n\tat org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:132)\n\tat org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:151)\n\tat java.sql.DriverManager.getConnection(DriverManager.java:664)\n\tat java.sql.DriverManager.getConnection(DriverManager.java:208)\n\tat org.apache.calcite.avatica.jdbc.JdbcMeta.openConnection(JdbcMeta.java:576)\n\t... 15 more\nCaused by: java.io.IOException: java.lang.reflect.InvocationTargetException\n\tat org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)\n\tat org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427)\n\tat org.apache.hadoop.hbase.client.ConnectionManager.createConnectionInternal(ConnectionManager.java:336)\n\tat org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:144)\n\tat org.apache.phoenix.query.HConnectionFactory$HConnectionFactoryImpl.createConnection(HConnectionFactory.java:47)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:285)\n\t... 26 more\nCaused by: java.lang.reflect.InvocationTargetException\n\tat sun.reflect.GeneratedConstructorAccessor6.newInstance(Unknown Source)\n\tat sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)\n\tat java.lang.reflect.Constructor.newInstance(Constructor.java:422)\n\tat org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)\n\t... 31 more\nCaused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil\n\tat org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)\n\tat org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)\n\tat org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)\n\tat org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:886)\n\tat org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:642)\n\t... 35 more\n\22\364\2RuntimeException: java.sql.SQLException: ERROR 103 (08004): Unable to establish connection. -> SQLException: ERROR 103 (08004): Unable to establish connection. -> IOException: java.lang.reflect.InvocationTargetException -> InvocationTargetException: (null exception message) -> NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil \377\377\377\377\17*\005000002\35\n\33gdoop-pqs-server-staging:87658\1", 8192, 0, NULL, NULL) = 5156
write(2, "AvaticaClientRuntimeException: Remote driver error: RuntimeException: java.sql.SQLException: ERROR 103 (08004): Unable to establish connection. -> SQLException: ERROR 103 (08004): Unable to establish connection. -> IOException: java.lang.reflect.InvocationTargetException -> InvocationTargetException: (null exception message) -> NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil. Error -1 (00000) null\n\njava.lang.RuntimeException: java.sql.SQLException: ERROR 103 (08004): Unable to establish connection.\n\tat org.apache.calcite.avatica.jdbc.JdbcMeta.openConnection(JdbcMeta.java:579)\n\tat org.apache.calcite.avatica.remote.LocalService.apply(LocalService.java:263)\n\tat org.apache.calcite.avatica.remote.Service$OpenConnectionRequest.accept(Service.java:2026)\n\tat org.apache.calcite.avatica.remote.Service$OpenConnectionRequest.accept(Service.java:2009)\n\tat org.apache.calcite.avatica.remote.AbstractHandler.apply(AbstractHandler.java:102)\n\tat org.apache.calcite.avatica.remote.ProtobufHandler.apply(ProtobufHandler.java:38)\n\tat org.apache.calcite.avatica.server.AvaticaProtobufHandler.handle(AvaticaProtobufHandler.java:68)\n\tat org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:497)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:245)\n\tat org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)\n\tat java.lang.Thread.run(Thread.java:745)\nCaused by: java.sql.SQLException: ERROR 103 (08004): Unable to establish connection.\n\tat org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:395)\n\tat org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:287)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl.access$300(ConnectionQueryServicesImpl.java:170)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:1840)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:1819)\n\tat org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:77)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:1819)\n\tat org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:180)\n\tat org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:132)\n\tat org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:151)\n\tat java.sql.DriverManager.getConnection(DriverManager.java:664)\n\tat java.sql.DriverManager.getConnection(DriverManager.java:208)\n\tat org.apache.calcite.avatica.jdbc.JdbcMeta.openConnection(JdbcMeta.java:576)\n\t... 15 more\nCaused by: java.io.IOException: java.lang.reflect.InvocationTargetException\n\tat org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)\n\tat org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427)\n\tat org.apache.hadoop.hbase.client.ConnectionManager.createConnectionInternal(ConnectionManager.java:336)\n\tat org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:144)\n\tat org.apache.phoenix.query.HConnectionFactory$HConnectionFactoryImpl.createConnection(HConnectionFactory.java:47)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:285)\n\t... 26 more\nCaused by: java.lang.reflect.InvocationTargetException\n\tat sun.reflect.GeneratedConstructorAccessor6.newInstance(Unknown Source)\n\tat sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)\n\tat java.lang.reflect.Constructor.newInstance(Constructor.java:422)\n\tat org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)\n\t... 31 more\nCaused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil\n\tat org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)\n\tat org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)\n\tat org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)\n\tat org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:886)\n\tat org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:642)\n\t... 35 more\n\n", 4948) = 4948
recvfrom(16, "HTTP/1.1 500 Server Error\r\nDate: Fri, 18 Nov 2016 21:19:05 GMT\r\nContent-Type: application/octet-stream;charset=utf-8\r\nContent-Length: 4983\r\nServer: Jetty(9.2.z-SNAPSHOT)\r\n\r\n\n8org.apache.calcite.avatica.proto.Responses$ErrorResponse\22\272&\n\222#java.lang.RuntimeException: java.sql.SQLException: ERROR 103 (08004): Unable to establish connection.\n\tat org.apache.calcite.avatica.jdbc.JdbcMeta.openConnection(JdbcMeta.java:579)\n\tat org.apache.calcite.avatica.remote.LocalService.apply(LocalService.java:263)\n\tat org.apache.calcite.avatica.remote.Service$OpenConnectionRequest.accept(Service.java:2026)\n\tat org.apache.calcite.avatica.remote.Service$OpenConnectionRequest.accept(Service.java:2009)\n\tat org.apache.calcite.avatica.remote.AbstractHandler.apply(AbstractHandler.java:102)\n\tat org.apache.calcite.avatica.remote.ProtobufHandler.apply(ProtobufHandler.java:38)\n\tat org.apache.calcite.avatica.server.AvaticaProtobufHandler.handle(AvaticaProtobufHandler.java:68)\n\tat org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:497)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:245)\n\tat org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)\n\tat java.lang.Thread.run(Thread.java:745)\nCaused by: java.sql.SQLException: ERROR 103 (08004): Unable to establish connection.\n\tat org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:395)\n\tat org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:287)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl.access$300(ConnectionQueryServicesImpl.java:170)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:1840)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:1819)\n\tat org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:77)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:1819)\n\tat org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:180)\n\tat org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:132)\n\tat org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:151)\n\tat java.sql.DriverManager.getConnection(DriverManager.java:664)\n\tat java.sql.DriverManager.getConnection(DriverManager.java:208)\n\tat org.apache.calcite.avatica.jdbc.JdbcMeta.openConnection(JdbcMeta.java:576)\n\t... 15 more\nCaused by: java.io.IOException: java.lang.reflect.InvocationTargetException\n\tat org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)\n\tat org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427)\n\tat org.apache.hadoop.hbase.client.ConnectionManager.createConnectionInternal(ConnectionManager.java:336)\n\tat org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:144)\n\tat org.apache.phoenix.query.HConnectionFactory$HConnectionFactoryImpl.createConnection(HConnectionFactory.java:47)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:285)\n\t... 26 more\nCaused by: java.lang.reflect.InvocationTargetException\n\tat sun.reflect.GeneratedConstructorAccessor6.newInstance(Unknown Source)\n\tat sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)\n\tat java.lang.reflect.Constructor.newInstance(Constructor.java:422)\n\tat org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)\n\t... 31 more\nCaused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil\n\tat org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)\n\tat org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)\n\tat org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)\n\tat org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:886)\n\tat org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:642)\n\t... 35 more\n\22\364\2RuntimeException: java.sql.SQLException: ERROR 103 (08004): Unable to establish connection. -> SQLException: ERROR 103 (08004): Unable to establish connection. -> IOException: java.lang.reflect.InvocationTargetException -> InvocationTargetException: (null exception message) -> NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil \377\377\377\377\17*\005000002\35\n\33gdoop-pqs-staging:87658\1", 8192, 0, NULL, NULL) = 5156
write(2, "AvaticaClientRuntimeException: Remote driver error: RuntimeException: java.sql.SQLException: ERROR 103 (08004): Unable to establish connection. -> SQLException: ERROR 103 (08004): Unable to establish connection. -> IOException: java.lang.reflect.InvocationTargetException -> InvocationTargetException: (null exception message) -> NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil. Error -1 (00000) null\n\njava.lang.RuntimeException: java.sql.SQLException: ERROR 103 (08004): Unable to establish connection.\n\tat org.apache.calcite.avatica.jdbc.JdbcMeta.openConnection(JdbcMeta.java:579)\n\tat org.apache.calcite.avatica.remote.LocalService.apply(LocalService.java:263)\n\tat org.apache.calcite.avatica.remote.Service$OpenConnectionRequest.accept(Service.java:2026)\n\tat org.apache.calcite.avatica.remote.Service$OpenConnectionRequest.accept(Service.java:2009)\n\tat org.apache.calcite.avatica.remote.AbstractHandler.apply(AbstractHandler.java:102)\n\tat org.apache.calcite.avatica.remote.ProtobufHandler.apply(ProtobufHandler.java:38)\n\tat org.apache.calcite.avatica.server.AvaticaProtobufHandler.handle(AvaticaProtobufHandler.java:68)\n\tat org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:497)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:245)\n\tat org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)\n\tat java.lang.Thread.run(Thread.java:745)\nCaused by: java.sql.SQLException: ERROR 103 (08004): Unable to establish connection.\n\tat org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:395)\n\tat org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:287)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl.access$300(ConnectionQueryServicesImpl.java:170)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:1840)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:1819)\n\tat org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:77)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:1819)\n\tat org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:180)\n\tat org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:132)\n\tat org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:151)\n\tat java.sql.DriverManager.getConnection(DriverManager.java:664)\n\tat java.sql.DriverManager.getConnection(DriverManager.java:208)\n\tat org.apache.calcite.avatica.jdbc.JdbcMeta.openConnection(JdbcMeta.java:576)\n\t... 15 more\nCaused by: java.io.IOException: java.lang.reflect.InvocationTargetException\n\tat org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)\n\tat org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:427)\n\tat org.apache.hadoop.hbase.client.ConnectionManager.createConnectionInternal(ConnectionManager.java:336)\n\tat org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:144)\n\tat org.apache.phoenix.query.HConnectionFactory$HConnectionFactoryImpl.createConnection(HConnectionFactory.java:47)\n\tat org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:285)\n\t... 26 more\nCaused by: java.lang.reflect.InvocationTargetException\n\tat sun.reflect.GeneratedConstructorAccessor6.newInstance(Unknown Source)\n\tat sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)\n\tat java.lang.reflect.Constructor.newInstance(Constructor.java:422)\n\tat org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)\n\t... 31 more\nCaused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil\n\tat org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)\n\tat org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)\n\tat org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)\n\tat org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:886)\n\tat org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:642)\n\t... 35 more\n\n", 4948) = 4948

[#5]

Yes, the Phoenix Query Server is going to act the same as a normal client. Have you changed anything on the system over a standard installation? PQS is unable to find a class which is should find in the hbase-protocol.jar

NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.protobuf.ProtobufUtil.

Please inspect the classpath of PQS and ensure that it is referencing the jar correctly.

[#6]

I don't see ProtobufUtil.class on hbase-protocol.jar.

karthick@pqs-client: strace $ jar -tf /usr/hdp/current/hbase-client/lib/phoenix-4.4.0-HBase-1.1-client.jar  | grep ProtobufUtil
org/apache/phoenix/protobuf/ProtobufUtil.class
org/apache/hadoop/hbase/protobuf/ProtobufUtil$1.class
org/apache/hadoop/hbase/protobuf/ProtobufUtil.class

So, my problem is that ProtobufUtil.class is actually present in my classpath but for somereason, phoenix queryServer is not able to load it. Another thing to note is that I am able to connect with sqlline.py. Its just the thin client that's throwing the exception I pasted.

[#7]

Sorry, yes, it looks like I was mistaken. In this version of HBase, the ProtobufUtil class is included in hbase-client.jar which is included in the phoenix-client.jar.

Have you actually verified that the jar is on the PQS's classpath? The thin-client is only displaying the error that the server returned. It is not encountering the error directly.

Also, you should really create your own question to address this issue instead of piggy-backing on this issue. They are not the same.

[#8]

Thanks for reply @Josh Elser. Will create a separate post with all the information I got.