Setting Up HA for Oozie with HAProxy

[#1]

Setting up High Availability for Oozie server.

Prerequisite: -

1. To have multiple Oozie instances we need to have either MYSql/Postgres/Oracle DB instead of the Dubey Database
2. We need to have HA Proxy /F5 Setup for Load Balancing. For Production I would suggest to get the F5 Load Balancer which should be managed by the Network Team.
Steps to setup Multiple oozie servers followed by HAProxy setup.
• Please find the attached Steps to add Multiple Oozie Instances
• https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-operations/content/adding_an_oozi...).

Setting up the ZK Quorum and oozie_base_url

Inorder to enable the HA feature of oozie we need to use zookeeper quorum and expose the Url of the Proxy server.

Oozie-site.xml

• ozie.zookeeper.connection.string provide the Zookeepr quorum in my lab e.g. "hwc3206-node2.hogwarts-labs.com:2181,hwc3206-node3.hogwarts-labs.com:2181,hwc3206-node4.hogwarts-labs.com:2181"
• oozie.services.ext add the class

  org.apache.oozie.service.ZKLocksService,
  org.apache.oozie.service.ZKXLogStreamingService,
  org.apache.oozie.service.ZKJobsConcurrencyService

• oozie.base.url (http://hwc3206-node1.hogwarts-labs.com:11000/oozie). this is the Proxy server will be used for all accepting all inward conncetion
Uncomment oozie_base_url section in oozie-env.

Finally restart the Oozie Service

Detailed Steps are documented in -> https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-operations/content/adding_an_oozi...

Setting up HA Proxy:-

• Install Haproxy server :- yum install haproxy
• Post installation it will it will create a .cfg file (/etc/haproxy/haproxy.cfg)
• Configure the Front_End and Back_end Nodes in the haproxy.cfg
Example of haproxy.cfg from my cluster

(Proxy servers are running on hwc3206-node1.hogwarts-labs.com and oozie Instances are running on hwc3206-node3.hogwarts-labs.com,hwc3206-node4.hogwarts-labs.com)

=======================================================

• #---------------------------------------------------------------------
• # main frontend which proxys to the backends
• #---------------------------------------------------------------------
• frontend hwc3206-node1.hogwarts-labs.com
• bind *:11000
• mode http
• # acl url_static path_beg -i /static /images /javascript /stylesheets
• # acl url_static path_end -i .jpg .gif .png .css .js
• # use_backend static if url_static
• default_backend oozie_servers
• #---------------------------------------------------------------------
• # static backend for serving up images, stylesheets and such
• #---------------------------------------------------------------------
• backend static
• balance roundrobin
• server static hwc3206-node3.hogwarts-labs.com:4331 check
• #---------------------------------------------------------------------
• # round robin balancing between the various backends
• #---------------------------------------------------------------------
• backend oozie_servers
• balance roundrobin
• server app1 hwc3206-node3.hogwarts-labs.com:11000 check
• server app2 hwc3206-node4.hogwarts-labs.com:11000 check
======================================================
• Finally start the HAProxy Server :- /etc/init.d/haproxy start

Validation of the HAProxy and Oozie :-

Please connect to the Oozie Web Console from the by using the Frontend/Proxy-server IP/Hostsnname

• http://172.25.39.28:11000/oozie/?user.name=admin
• http://hwc3206-node1.hogwarts-labs.com :11000/oozie/?user.name=admin (with Hostname)

Thanks

Sumit Sarkar