Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to Enable Zookeeper Discovery for HiveServer2 HA

Solved Go to solution
Highlighted

How to Enable Zookeeper Discovery for HiveServer2 HA

Hi,

I am trying to set up zookeeper discovery for HS2 and I am getting various connection errors when I try to connect via beeline.

The following is the connection string.

!connect jdbc:hive2://host1.com:2181,host2.com:2181,host3.com:2181;serviceDiscoveryMode=zooKeeper; zooKeeperNamespace=hiveserver2 

I am following this documentation. Can anyone confirm if there are any other steps/configs required? Is the connection string wrong perhaps? Does HS2 need to run in http mode or is binary ok?

Configuration Requirements

1. Set hive.zookeeper.quorum to the ZooKeeper ensemble (a comma separated list of ZooKeeper server host:ports running at the cluster)

2. Customize hive.zookeeper.session.timeout so that it closes the connection between the HiveServer2’s client and ZooKeeper if a heartbeat is not received within the timeout period.

3. Set hive.server2.support.dynamic.service.discovery to true

4. Set hive.server2.zookeeper.namespace to the value that you want to use as the root namespace on ZooKeeper. The default value is hiveserver2.

5. The adminstrator should ensure that the ZooKeeper service is running on the cluster, and that each HiveServer2 instance gets a unique host:port combination to bind to upon startup.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: How to Enable Zookeeper Discovery for HiveServer2 HA

Master Collaborator

Can you try the following connection url (observe the / after the <ZOOKEEPER QUORUM>)?

jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver

Above is for binary mode, for http mode

jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver;transportMode=http;httpPath=cliservice

For secure environments you will additionally have to add the hive principal, eg.

jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver;principal=hive/_HOST@EXAMPLE.COM;transportMode=http;httpPath=cliservice
12 REPLIES 12

Re: How to Enable Zookeeper Discovery for HiveServer2 HA

Master Collaborator

Can you try the following connection url (observe the / after the <ZOOKEEPER QUORUM>)?

jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver

Above is for binary mode, for http mode

jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver;transportMode=http;httpPath=cliservice

For secure environments you will additionally have to add the hive principal, eg.

jdbc:hive2://<ZOOKEEPER QUORUM>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver;principal=hive/_HOST@EXAMPLE.COM;transportMode=http;httpPath=cliservice

Re: How to Enable Zookeeper Discovery for HiveServer2 HA

I'll try it, thanks!

Re: How to Enable Zookeeper Discovery for HiveServer2 HA

Unfortunately that seems not to work. Does it make a difference to the URL if hiveserver2 is in http transport mode? You don't have to add that in somewhere like you do in a normal beeline connection string?

Re: How to Enable Zookeeper Discovery for HiveServer2 HA

Master Collaborator

Updated the answer to include the url for http mode as well as secure http mode. Beyond this there are other modes like ssl http, ldap, ldap http. For each one the URL is configured a little differently.

Re: How to Enable Zookeeper Discovery for HiveServer2 HA

works perfectly now, thanks!

Re: How to Enable Zookeeper Discovery for HiveServer2 HA

A minor correction: In HDP-2.3.4, the zooKeeperNamespace is called "hiveserver2", not just "hiveserver". With that fix it works great!

Re: How to Enable Zookeeper Discovery for HiveServer2 HA

Cloudera Employee

With HDP 2.3.4 and later releases, you don't need to specify any additional options as it includes the changes in HIVE-11581. Just the first URL is sufficient - jdbc:hive2://<zookeeper quorum>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2.

See the hiveserver2 client doc

Also make sure you have the url within quotes if you are specifying it in commandline. ie, beeline -u 'hive2...'

Re: How to Enable Zookeeper Discovery for HiveServer2 HA

This should work

beeline -u "jdbc:hive2://zk1:2181,zk3:2181,zk2:2181/default;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_HOST@REALM”

non kerberos 


beeline -u "jdbc:hive2://zk1:2181,zk3:2181,zk2:2181/default;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2

Re: How to Enable Zookeeper Discovery for HiveServer2 HA

Contributor

Can we specify queue to be used instead of default queue? Like append "tez.queue.name=xyz" to the command. Will it work?

Don't have an account?
Coming from Hortonworks? Activate your account here