CDH 5.4.0 and HiveServer2 High Availability via ZooKeeper

[#1]

Hi

Does CDH 5.4.2 support HiveServer2 high availability? https://issues.apache.org/jira/browse/HIVE-7935

jdbc:hive2://<zookeeperServer1>:port,<zookeeperServer2>:port,<zookeeperServer3>:port/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2

I cant find any references in http://www.cloudera.com/content/www/en-us/documentation/cdh/5-0-x/CDH5-Installation-Guide/cdh5ig_hiv...

Regards

Deenar

[#2]

like hiveserver2 do not register it's self to zookeeper, but how to config it ?

[#3]

i got it.

from http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_hadoop-ha/content/ha-hs2-rolling-upgrade....

use cloudera manager:

1、open hive -> Configuration -> Category -> Advanced
2、find "HiveServer2 Advanced Configuration Snippet (Safety Valve) for hive-site.xml"
3、write:
<property>
<name>hive.server2.support.dynamic.service.discovery</name>
<value>true</value>
</property>
4、restart and hiveserver2 is register to zookeeper.

[#4]

Have you tried it on your cluster and does it work fine. I know about this property. I was wondering if this supported on CDH 5.4.0 ?

[#5]

sorry, i only tried it with cdh5.4.8 and i don't have cdh5.4.0 cluster.

[#6]

Hi,
I tried this on CDH 5.4.4 and it works.
Actually, I just tried the jdbc string providing the zookeeper ensamble, I configured the CM property in advance and it just worked.

Imho this property is very useful, I don't know why Cloudera is not supporting it.

[#7]

Thanks guys

[#8]

I am a PM working on Hive at Cloudera. Since HS2 HA (dynamic service discovery) has significant known issues, we don't consider it production ready or supported as specified in release notes for CDH 5.4. Therefore, this experimental feature is not supported in any CDH release.

For our customers, Hive workloads often constitute the backbone of their data infrastructure. Consequently, stability and reliability of Hive components are of utmost importance. Customers should use an experimental feature e.g. HS2 HA only under the guidance of our technical experts. Customers have access to internal reference docs for such features. If you are a customer, please contact your Cloudera account manager for further details and technical guidance.

[#9]

This is a good answer !

I work for a paying Cloudera Customer, I am experimenting new features to propose future enhancements. I was experimenting with HA HS2, and was choosing over separate HS2 servers (externally balanced via HAProxy or Netscaler) or using this way. OK, separate HS2 are not proper HA, but sort of.

Since this sounded better to me, just because we are using zookeeper and not an external software, I was searching for info.

Anyway, do you have any foreseen about CDH version supporting this ? Would it be in any problem with Metastore HA + Sentry service + CM ACL plugin, which is also unsupported ?

I asked to the support and they offer to send me the local support agent: I don't want support for implementing it, I just want to know if it makes sense for me to investigate it, since it will supported soon. This way I can tell my client that experimentation makes sense, hope you understand what I mean.

Bye

Omar

[#10]

Omar,

In general, separate HS2 is not the right way for HA. It's less efficient. However, a lot depends on your context. Since you are working for a Cloudera customer, please approach your customer's account manager/sales engineers. They would have the required context to help you. 

Supporting HS2 HA is one of our top priorities. Therefore, it will be available soon, though I cannot commit to a version or timeline yet. Let me know if you have any specific timeline in mind. CDH is a platform and therefore, we will also ensure seamless integration of HS2 HA with HMS, Sentry (our auth/acl service) as well as other relevant components. 

Therefore, unless you have a short timeline requirements, it makes sense for you to explore this feature. Please contact account teams for channeling further inquiry regarding this feature.