03-30-2015 05:15 AM
Is there any way to force a submitted coordinator to pick up updated definition of the workflow?
E.g. in a simple workflow with one action which runs a Hive script, if that script is updated, currently I see the only option is to kill the submitted coordinator and submit it again. Is there any other way in which it could pick up the modified Hive script?
03-30-2015 05:22 AM
03-30-2015 05:46 AM
Hi thanks for the quick reply, I thought I saw different behaviour but it might have been my mistake. Will double check and raise support call if the expected behaviour is as you decribed and I am seeing something different.
03-30-2015 05:49 AM
Also just to confirm - given the behaviour described in the following thread to which I just replied, workflow.xml is not updated in the workspace directory immediately when saved in Hue (as per the response it is only updated when first submitted). So whilst I have made changes to workflow definition (added extra step), the workflow.xml file is not updated in the folder. Is it then expected that the submitted coordinator will pick up those changes?
03-30-2015 08:54 AM
To follow up: the behaviour I observed is that after the scheduled execution through coordinator, the new actions were shown in the Workflows dashboard Graph tab, however they were skipped during the execution run. See attached screenshot. The new actions introduced were the fork "RunThingsInParallel" and the two actions (both Hive) in the right hand side branch. They are all greyed out rather than the green OK status during and after the execution, which was successful. I will be raising a support call for this.
04-25-2015 08:52 PM
For the benefit of community members: Cloudera support team confirmed that an updated workflow definition is NOT picked up by an existing submitted coordinator schedule. The workarounds are either to kill the coordinator schedule and resubmit, or to submit the workflow manually which will force the workflow.xml to be updated.
04-27-2015 11:13 AM