Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

hive connection via zookeeper

avatar
Contributor

Hi, 

I'm trying to connect to hive from zookeeper, I'm getting a java null pointer exception with beeline cmd. currently using CDH 5.16.2 version. 

I have a valid kerberos ticket:

beeline --verbose=true -u "jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveServer2;transportMode=binary;httpPath=cliservice;"
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
issuing: !connect jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveServer2;transportMode=binary;httpPath=cliservice; '' [passwd stripped]
scan complete in 2ms
Connecting to jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveServer2;transportMode=binary;httpPath=cliservice;
java.lang.NullPointerException
at org.apache.thrift.transport.TSocket.open(TSocket.java:209)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:266)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:203)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:168)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:146)
at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:211)
at org.apache.hive.beeline.Commands.connect(Commands.java:1529)
at org.apache.hive.beeline.Commands.connect(Commands.java:1424)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:52)
at org.apache.hive.beeline.BeeLine.execCommandWithPrefix(BeeLine.java:1139)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1178)
at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:818)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:898)
at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:518)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:501)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:226)
at org.apache.hadoop.util.RunJar.main(RunJar.java:141)
Beeline version 1.1.0-cdh5.16.2 by Apache Hive
0: jdbc:hive2://ZK1:2181, (closed)>

12 REPLIES 12

avatar
Explorer

Try and use it after this modification. I succeeded like this.

[You]

jdbc:hive2://zk=hadoopcluster01:2181,hadoopcluster02:2181/hiveserver2

[My Proposal]

jdbc:hive2://zk=hadoopcluster01:2181/hiveserver2,hadoopcluster02:2181/hiveserver2

avatar
Contributor

Thanks Lewis, but it didn't worked!!

 

Caused by: java.lang.IllegalArgumentException: A HostProvider may not be empty!
at org.apache.zookeeper.client.StaticHostProvider.<init>(StaticHostProvider.java:72)
at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:445)
at org.apache.curator.utils.DefaultZookeeperFactory.newZooKeeper(DefaultZookeeperFactory.java:29)
at org.apache.curator.framework.imps.CuratorFrameworkImpl$2.newZooKeeper(CuratorFrameworkImpl.java:154)
at org.apache.curator.HandleHolder$1.getZooKeeper(HandleHolder.java:94)
at org.apache.curator.HandleHolder.internalClose(HandleHolder.java:128)
at org.apache.curator.HandleHolder.closeAndClear(HandleHolder.java:71)
at org.apache.curator.ConnectionState.close(ConnectionState.java:114)
... 30 more
Error: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 configs from ZooKeeper (state=,code=0)
Beeline version 1.1.0-cdh5.16.2 by Apache Hive
beeline>

avatar
Expert Contributor

@raghu9raghavend 

 

I see that you are using the below connect string:

 

jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveServer2;transportMode=binary;httpPath=cliservice;

 

You are using both "transportMode=binary" and then you are providing the "httpPath=cliservice" as well.

"httpPath=cliservice" should be provided only when "transportMode=http".

Thus, if your Hiveserver2 is running in HTTP mode, please set the "transportMode=http" in the connect string.

 

jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveServer2;transportMode=http;httpPath=cliservice;

 

Alternatively, if you are have HiveServer2 in binary mode, remove both the options.

 

jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveServer2;

avatar
Contributor

Thanks Rohit, I tried both options but it did not helped me. 

 

Apparently cloudera engineer is saying that this option(i.e, connecting to hive via zookeeper) will not be supported in CDH 5.16.2 version. Is it true?

beeline -u "jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hive_zookeeper_namespace_hive;transportMode=http;httpPath=cliservice;"
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
scan complete in 1ms
Connecting to jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hive_zookeeper_namespace_hive;transportMode=http;httpPath=cliservice;
20/01/17 09:27:26 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
20/01/17 09:27:27 [main]: INFO jdbc.HiveConnection: Failed to connect to null:0
Error: Could not open client transport for any of the Server URI's in ZooKeeper: Unable to read HiveServer2 configs from ZooKeeper (state=08S01,code=0)
Beeline version 1.1.0-cdh5.16.2 by Apache Hive
beeline>

 

beeline -u "jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hive_zookeeper_namespace_hive;"
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
scan complete in 1ms
Connecting to jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hive_zookeeper_namespace_hive;
java.lang.NullPointerException
Beeline version 1.1.0-cdh5.16.2 by Apache Hive
0: jdbc:hive2://ZK1:2181, (closed)>

 

Thanks,

Raghu.

avatar
New Contributor

Hi, do you know this error reason and how to fix it?I have the same error

avatar
Contributor

Hi Raghu,

 

I think we have missed the service discovery 

Please check with the below command and confirm

 

Host names and port numbers  where your zookeeper is running . We also call it as Zookeeper ensemble.
We can easily get this value from property hive.zookeeper.quorum in Hive.

zookeeper's default port number is 2181. You can get zookeeper host names from Zookeeper configuration files also.


Specify service discovery mode using serviceDiscoveryMode=zooKeeper .


Specify zookeeper namespace as hivesever2. This is the value of hive.server2.zookeeper.namespace property in Hive.


We are using below connection string .

jdbc:hive2://datanode1:2181,master1:2181,master2:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2

avatar
New Contributor

Thanks!

avatar
Guru

@raghu9raghavend  I think I caught the issue, you are using binary and proving httppath

 

jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveServer2;transportMode=binary;httpPath=cliservice; 

 

Please check,if below set to http or binary 

hive.server2.transport.mode

Change the url:

 

http:

jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;transportMode=http;httpPath=cliservice; 

 

binary:

jdbc:hive2://ZK1:2181,ZK2:2181,ZK3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2

Change 

 hiveSever2 ==> hiveserver2

avatar
Contributor

Thanks for the responses. This is a old one and resolved long back.