Reply
New Contributor
Posts: 4
Registered: ‎04-28-2016

Externalize properties for Oozie workflows

Hi,

we are developing dozens of different workflows in Oozie. We use 3 environments, DEV, PRE and PRO. 

 

What I'd like to do is to keep same workflow.xml files in all 3 environments and only read properties for db connections, urls, authentications etc from the Cloudera environment. In this way, it would be possible to version control the files and deploy them easily. Is it possible? 

 

Googling, I saw maybe config-default.xml would be a place for default property values, but I cannot find any in our Cloudera 5.10. We use Hue to design the workflows, algthough we can use other way to define WFs. 

 

Thanks!

Cloudera Employee
Posts: 213
Registered: ‎03-23-2015

Re: Externalize properties for Oozie workflows

Hi,

I think it should be doable, though not through Hue.

You can try to put the environment specific variables into job.properties file and pass them into the workflow, so that the workflow is generic and can be used in other environments.

Hue will hard code things like credentials into the workflow if you are in secured environment, so it won't work. You might have to try to manually create those workflows and submit jobs from command line.
New Contributor
Posts: 4
Registered: ‎04-28-2016

Re: Externalize properties for Oozie workflows

Hi Eric,

thanks for the reply. I think I saw the cmd line example you refer to, is it this?  https://oozie.apache.org/docs/4.0.0/DG_Examples.html

The thing is that our client wants to have some graphical interface like Hue, so they could control (including launching) any job in the datalake.

Maybe we could edit externally job.properties in the WF's workspace? Actually I've tried to edit the contents of the file, but Hue ignores the new values - when submitting, a popup opens with all the properties with values unchanged. Maybe I'm doing something wrong?

Cloudera Employee
Posts: 213
Registered: ‎03-23-2015

Re: Externalize properties for Oozie workflows

You can't modify workflow setting outside of Hue, including job.properties and workflow.xml, as Hue will re-generate them when you submit the workflow again.

If you want to stick with Hue, I do not think it will work. But let me check for you if Hue has such feature on the road map.
Highlighted
New Contributor
Posts: 4
Registered: ‎04-28-2016

Re: Externalize properties for Oozie workflows

[ Edited ]

Actually I edited the job.properties with Hue's file browser. But those original properties has to be stored somewhere else (in Hue's memory?) because they remain unchanged. I even restarted Oozie and Hue and reopened the WF, but the Hue still did not took into account new values of the job.properties.

Thanks.

Announcements