Support Questions

Find answers, ask questions, and share your expertise

How to setup High Availability for Oozie?

avatar

To have Oozie server in HA, it is mentioned in the Hortonworks documentation that it needs a Loadbalancer, Virtual IP, or Round-Robin DNS. As this is not part of Hadoop ecosystem, what tool is suggest to use here? HAProxy/nginx/or any other commercial one?

1 ACCEPTED SOLUTION

avatar

Ambari does not manage HA for Oozie yet. Here are some list of manual steps which I recently dug out for someone (AMBARI-6683 is the related JIRA but BUG-13082 has the relevant details you are looking for)

Pasting here:

1) Added oozie-server component, using +Add button on host page.

2) Using apache httpd(using mod_proxy and mod_proxy_balancer), configured load balancing with url liveness check. It means, that returned url for oozie previously checked for availability. We need this, because one of oozie can be unavailable, so load balancer should not return url for it.

3) In oozie-site.xml config:

– add oozie.zookeeper.connection.string = <list of zookeeper hosts with ports> (example: c6401.ambari.apache.org:2181,c6402.ambari.apache.org:2181,c6403.ambari.apache.org:2181)

– add these classes "org.apache.oozie.service.ZKLocksService,org.apache.oozie.service.ZKXLogStreamingService,org.apache.oozie.service.ZKJobsConcurrencyService" to property oozie.services.ext.

– change oozie.base.url to http://<loadbalancer_hostname>:11000/oozie

4) In oozie-env.sh config:

– uncomment OOZIE_BASE_URL property and change value to point to the loadbalancer (example of value: http://<loadbalancer_hostname>:11000/oozie)

5) In core-site.xml:

– add host with newly added oozie-server to hadoop.proxyuser.oozie.hosts property. Hosts should be comma separated.

6) Restart all needed services.

Note1: Oozie HA will work only for existing db, because as i know, derby db doesn't support concurrent connections.

View solution in original post

10 REPLIES 10

avatar
Master Guru

Step by step guide to configure Oozie HA in kerberized environment

http://crazyadmins.com/tag/oozie-ha-loadbalancer/