hive connection via zookeeper

[#1]

Hi, 

I'm trying to connect to hive from zookeeper, I'm getting a java null pointer exception with beeline cmd. currently using CDH 5.16.2 version. 

I have a valid kerberos ticket:

beeline --verbose=true -u "jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveServer2;transportMode=binary;httpPath=cliservice;"
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
issuing: !connect jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveServer2;transportMode=binary;httpPath=cliservice; '' [passwd stripped]
scan complete in 2ms
Connecting to jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveServer2;transportMode=binary;httpPath=cliservice;
java.lang.NullPointerException
at org.apache.thrift.transport.TSocket.open(TSocket.java:209)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:266)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:203)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:168)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:146)
at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:211)
at org.apache.hive.beeline.Commands.connect(Commands.java:1529)
at org.apache.hive.beeline.Commands.connect(Commands.java:1424)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:52)
at org.apache.hive.beeline.BeeLine.execCommandWithPrefix(BeeLine.java:1139)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1178)
at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:818)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:898)
at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:518)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:501)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:226)
at org.apache.hadoop.util.RunJar.main(RunJar.java:141)
Beeline version 1.1.0-cdh5.16.2 by Apache Hive
0: jdbc:hive2://ZK1:2181, (closed)>

[#2]

Try and use it after this modification. I succeeded like this.

[You]

jdbc:hive2://zk=hadoopcluster01:2181,hadoopcluster02:2181/hiveserver2

[My Proposal]

jdbc:hive2://zk=hadoopcluster01:2181/hiveserver2,hadoopcluster02:2181/hiveserver2

[#3]

Thanks Lewis, but it didn't worked!!

Caused by: java.lang.IllegalArgumentException: A HostProvider may not be empty!
at org.apache.zookeeper.client.StaticHostProvider.<init>(StaticHostProvider.java:72)
at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:445)
at org.apache.curator.utils.DefaultZookeeperFactory.newZooKeeper(DefaultZookeeperFactory.java:29)
at org.apache.curator.framework.imps.CuratorFrameworkImpl$2.newZooKeeper(CuratorFrameworkImpl.java:154)
at org.apache.curator.HandleHolder$1.getZooKeeper(HandleHolder.java:94)
at org.apache.curator.HandleHolder.internalClose(HandleHolder.java:128)
at org.apache.curator.HandleHolder.closeAndClear(HandleHolder.java:71)
at org.apache.curator.ConnectionState.close(ConnectionState.java:114)
... 30 more
Error: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 configs from ZooKeeper (state=,code=0)
Beeline version 1.1.0-cdh5.16.2 by Apache Hive
beeline>

[#4]

@raghu9raghavend 

I see that you are using the below connect string:

jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveServer2;transportMode=binary;httpPath=cliservice;

You are using both "transportMode=binary" and then you are providing the "httpPath=cliservice" as well.

"httpPath=cliservice" should be provided only when "transportMode=http".

Thus, if your Hiveserver2 is running in HTTP mode, please set the "transportMode=http" in the connect string.

jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveServer2;transportMode=http;httpPath=cliservice;

Alternatively, if you are have HiveServer2 in binary mode, remove both the options.

jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveServer2;

[#5]

Thanks Rohit, I tried both options but it did not helped me. 

Apparently cloudera engineer is saying that this option(i.e, connecting to hive via zookeeper) will not be supported in CDH 5.16.2 version. Is it true?

beeline -u "jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hive_zookeeper_namespace_hive;transportMode=http;httpPath=cliservice;"
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
scan complete in 1ms
Connecting to jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hive_zookeeper_namespace_hive;transportMode=http;httpPath=cliservice;
20/01/17 09:27:26 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
Error: Could not open client transport for any of the Server URI's in ZooKeeper: Unable to read HiveServer2 configs from ZooKeeper (state=08S01,code=0)
Beeline version 1.1.0-cdh5.16.2 by Apache Hive
beeline>

beeline -u "jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hive_zookeeper_namespace_hive;"
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
scan complete in 1ms
Connecting to jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hive_zookeeper_namespace_hive;
java.lang.NullPointerException
Beeline version 1.1.0-cdh5.16.2 by Apache Hive
0: jdbc:hive2://ZK1:2181, (closed)>

Thanks,

Raghu.