Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (2)
Not applicable

Setting up High Availability for Oozie server.

Prerequisite: -

  1. To have multiple Oozie instances we need to have either MYSql/Postgres/Oracle DB instead of the Dubey Database
  2. We need to have HA Proxy /F5 Setup for Load Balancing. For Production I would suggest to get the F5 Load Balancer which should be managed by the Network Team.
  3. Steps to setup Multiple oozie servers followed by HAProxy setup.

      Setting up the ZK Quorum and oozie_base_url

      Inorder to enable the HA feature of oozie we need to use zookeeper quorum and expose the Url of the Proxy server.

      Oozie-site.xml

      • ozie.zookeeper.connection.string provide the Zookeepr quorum in my lab e.g. "hwc3206-node2.hogwarts-labs.com:2181,hwc3206-node3.hogwarts-labs.com:2181,hwc3206-node4.hogwarts-labs.com:2181"
      • oozie.services.ext add the class
        org.apache.oozie.service.ZKLocksService,
        org.apache.oozie.service.ZKXLogStreamingService,
        org.apache.oozie.service.ZKJobsConcurrencyService
      • oozie.base.url (http://hwc3206-node1.hogwarts-labs.com:11000/oozie). this is the Proxy server will be used for all accepting all inward conncetion
      • Uncomment oozie_base_url section in oozie-env.

        Finally restart the Oozie Service

        Detailed Steps are documented in -> https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-operations/content/adding_an_oozi...

        Setting up HA Proxy:-

        • Install Haproxy server :- yum install haproxy
        • Post installation it will it will create a .cfg file (/etc/haproxy/haproxy.cfg)
        • Configure the Front_End and Back_end Nodes in the haproxy.cfg
        • Example of haproxy.cfg from my cluster

          (Proxy servers are running on hwc3206-node1.hogwarts-labs.com and oozie Instances are running on hwc3206-node3.hogwarts-labs.com,hwc3206-node4.hogwarts-labs.com)

          =======================================================

          • #---------------------------------------------------------------------
          • # main frontend which proxys to the backends
          • #---------------------------------------------------------------------
          • frontend hwc3206-node1.hogwarts-labs.com
          • bind *:11000
          • mode http
          • # acl url_static path_beg -i /static /images /javascript /stylesheets
          • # acl url_static path_end -i .jpg .gif .png .css .js
          • # use_backend static if url_static
          • default_backend oozie_servers
          • #---------------------------------------------------------------------
          • # static backend for serving up images, stylesheets and such
          • #---------------------------------------------------------------------
          • backend static
          • balance roundrobin
          • server static hwc3206-node3.hogwarts-labs.com:4331 check
          • #---------------------------------------------------------------------
          • # round robin balancing between the various backends
          • #---------------------------------------------------------------------
          • backend oozie_servers
          • balance roundrobin
          • server app1 hwc3206-node3.hogwarts-labs.com:11000 check
          • server app2 hwc3206-node4.hogwarts-labs.com:11000 check
          • ======================================================

            • Finally start the HAProxy Server :- /etc/init.d/haproxy start


            Validation of the HAProxy and Oozie :-

            Please connect to the Oozie Web Console from the by using the Frontend/Proxy-server IP/Hostsnname

            Thanks

            Sumit Sarkar

    643 Views
    Don't have an account?
    Coming from Hortonworks? Activate your account here
    Version history
    Revision #:
    1 of 1
    Last update:
    ‎09-07-2018 01:13 PM
    Updated by:
     
    Contributors
    Top Kudoed Authors