question Re: Oozie with HDFS High Availability in Archives of Support Questions (Read Only)

Oozie with HDFS High Availability

ludof — Fri, 16 Sep 2022 13:12:25 GMT

Hello everyone,

I have a cluster with HDFS High Availability (HA) enabled. The cluster has two NameNode, one active and on in standby state, plus 3 journal nodes, a balancer and failover controllers.

My question: how should I configure Oozie workflows for nameNode and jobTracker parameters in job.properties file in order to point always to the active NameNode and JobTracker (in case of a failure or a manual switch of the NameNode)?

Thanks for any information

Re: Oozie with HDFS High Availability

Harsh J — Mon, 14 May 2018 16:12:52 GMT

The requirement for Oozie is not different than the general requirement
that after you enable HDFS HA (or YARN HA, etc.), always use the logical
URI everywhere and never directly place/hardcode a NameNode hostname in any
manual configuration.

Oozie as a service carries HDFS client configs that are maintained for it
by CM. These become HA-aware when you complete the HDFS HA wizard. All that
remains is that you submit the new jobs to Oozie with the nameNode and
jobTracker URIs pointing to the logical name (such as hdfs://nameservice1)
instead of the previous single-host/port value.

Re: Oozie with HDFS High Availability

ludof — Mon, 14 May 2018 17:28:41 GMT

Thanks @Harsh J, indeed I've finally solved using hdfs://hanameservice for name node and yarnrm for the job tracker.