Force coordinator to pick up updated workflow definition

[#1]

Hello,

Is there any way to force a submitted coordinator to pick up updated definition of the workflow?

E.g. in a simple workflow with one action which runs a Hive script, if that script is updated, currently I see the only option is to kill the submitted coordinator and submit it again. Is there any other way in which it could pick up the modified Hive script?

Thanks,

Martin

[#2]

If you update the workflow.xml at the same location the coordinator is
configured to look for it, it will automatically pick up the newer XML
contents for the future workflow instances.

[#3]

Hi thanks for the quick reply, I thought I saw different behaviour but it might have been my mistake. Will double check and raise support call if the expected behaviour is as you decribed and I am seeing something different.

Thanks,

Martin

[#4]

Also just to confirm - given the behaviour described in the following thread to which I just replied, workflow.xml is not updated in the workspace directory immediately when saved in Hue (as per the response it is only updated when first submitted). So whilst I have made changes to workflow definition (added extra step), the workflow.xml file is not updated in the folder. Is it then expected that the submitted coordinator will pick up those changes?

Thanks,

Martin

[#5]

To follow up: the behaviour I observed is that after the scheduled execution through coordinator, the new actions were shown in the Workflows dashboard Graph tab, however they were skipped during the execution run. See attached screenshot. The new actions introduced were the fork "RunThingsInParallel" and the two actions (both Hive) in the right hand side branch. They are all greyed out rather than the green OK status during and after the execution, which was successful. I will be raising a support call for this.

[#6]

For the benefit of community members: Cloudera support team confirmed that an updated workflow definition is NOT picked up by an existing submitted coordinator schedule. The workarounds are either to kill the coordinator schedule and resubmit, or to submit the workflow manually which will force the workflow.xml to be updated.

Martin

[#7 · Accepted solution]

This is JIRA HUE-2551 The two options currently available are - kill the coordinator and create a new one - run the workflow manually after the change.