Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to Enable Zookeeper Discovery for HiveServer2 HA

avatar

Hi,

I am trying to set up zookeeper discovery for HS2 and I am getting various connection errors when I try to connect via beeline.

The following is the connection string.

!connect jdbc:hive2://host1.com:2181,host2.com:2181,host3.com:2181;serviceDiscoveryMode=zooKeeper; zooKeeperNamespace=hiveserver2 

I am following this documentation. Can anyone confirm if there are any other steps/configs required? Is the connection string wrong perhaps? Does HS2 need to run in http mode or is binary ok?

Configuration Requirements

1. Set hive.zookeeper.quorum to the ZooKeeper ensemble (a comma separated list of ZooKeeper server host:ports running at the cluster)

2. Customize hive.zookeeper.session.timeout so that it closes the connection between the HiveServer2’s client and ZooKeeper if a heartbeat is not received within the timeout period.

3. Set hive.server2.support.dynamic.service.discovery to true

4. Set hive.server2.zookeeper.namespace to the value that you want to use as the root namespace on ZooKeeper. The default value is hiveserver2.

5. The adminstrator should ensure that the ZooKeeper service is running on the cluster, and that each HiveServer2 instance gets a unique host:port combination to bind to upon startup.

1 ACCEPTED SOLUTION

avatar

Can you try the following connection url (observe the / after the <ZOOKEEPER QUORUM>)?

jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver

Above is for binary mode, for http mode

jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver;transportMode=http;httpPath=cliservice

For secure environments you will additionally have to add the hive principal, eg.

jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver;principal=hive/_HOST@EXAMPLE.COM;transportMode=http;httpPath=cliservice

View solution in original post

13 REPLIES 13

avatar

Can you try the following connection url (observe the / after the <ZOOKEEPER QUORUM>)?

jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver

Above is for binary mode, for http mode

jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver;transportMode=http;httpPath=cliservice

For secure environments you will additionally have to add the hive principal, eg.

jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver;principal=hive/_HOST@EXAMPLE.COM;transportMode=http;httpPath=cliservice

avatar

I'll try it, thanks!

avatar

Unfortunately that seems not to work. Does it make a difference to the URL if hiveserver2 is in http transport mode? You don't have to add that in somewhere like you do in a normal beeline connection string?

avatar

Updated the answer to include the url for http mode as well as secure http mode. Beyond this there are other modes like ssl http, ldap, ldap http. For each one the URL is configured a little differently.

avatar

works perfectly now, thanks!

avatar
Master Guru

A minor correction: In HDP-2.3.4, the zooKeeperNamespace is called "hiveserver2", not just "hiveserver". With that fix it works great!

avatar
Contributor

With HDP 2.3.4 and later releases, you don't need to specify any additional options as it includes the changes in HIVE-11581. Just the first URL is sufficient - jdbc:hive2://<zookeeper quorum>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2.

See the hiveserver2 client doc

Also make sure you have the url within quotes if you are specifying it in commandline. ie, beeline -u 'hive2...'

avatar
Master Mentor

This should work

beeline -u "jdbc:hive2://zk1:2181,zk3:2181,zk2:2181/default;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_HOST@REALM”

non kerberos 


beeline -u "jdbc:hive2://zk1:2181,zk3:2181,zk2:2181/default;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2

avatar
Rising Star

Can we specify queue to be used instead of default queue? Like append "tez.queue.name=xyz" to the command. Will it work?