Load Balance Zookeeper using HAProxy

[#1]

This is the process to load balance zookeeper's behind a haproxy LB and the can be used for any other components as well .

1) Install HAproxy

yum install haproxy

2) Edit /etc/haproxy/haproxy.cfg and append the below config

#This specifies port which the end users will use to access zookeeper.

listen appname 0.0.0.0:80

#Specifies the access method tcp/http/udp , should be tcp for zookeeper

mode tcp

#Algorithm used for balancing requests

balance roundrobin

#It forwards the client ip requesting the service to zookeeper

option forwardfor

#The zookeeper server list and port

server mshaik-2 172.26.81.203:2181 check

server mshaik-3 172.26.81.204:2181 check

server mshaik-4 172.26.81.206:2181 check

3) Reload haproxy config file

haproxy -f /etc/haproxy.cfg -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)

4) Test the connection through beeline

beeline> !connect jdbc:hive2://mshaik-1.htworks.com:80/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2

Connecting to jdbc:hive2://mshaik-1.htworks.com:80/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2

Enter username for jdbc:hive2://mshaik-1.htworks.com:80/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2:

Enter password for jdbc:hive2://mshaik-1.htworks.com:80/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2:

Connected to: Apache Hive (version 1.2.1000.2.5.5.5-2)

Driver: Hive JDBC (version 1.2.1000.2.5.5.0-157)

Transaction isolation: TRANSACTION_REPEATABLE_READ

0: jdbc:hive2://mshaik-1.htworks.com:80/>