Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Force coordinator to pick up updated workflow definition

SOLVED Go to solution

Force coordinator to pick up updated workflow definition

Rising Star

Hello,

Is there any way to force a submitted coordinator to pick up updated definition of the workflow?

 

E.g. in a simple workflow with one action which runs a Hive script, if that script is updated, currently I see the only option is to kill the submitted coordinator and submit it again. Is there any other way in which it could pick up the modified Hive script?

 

Thanks,

Martin

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Force coordinator to pick up updated workflow definition

Cloudera Employee
This is JIRA HUE-2551 The two options currently available are - kill the coordinator and create a new one - run the workflow manually after the change.
6 REPLIES 6

Re: Force coordinator to pick up updated workflow definition

Master Guru
If you update the workflow.xml at the same location the coordinator is
configured to look for it, it will automatically pick up the newer XML
contents for the future workflow instances.

Re: Force coordinator to pick up updated workflow definition

Rising Star

Hi thanks for the quick reply, I thought I saw different behaviour but it might have been my mistake. Will double check and raise support call if the expected behaviour is as you decribed and I am seeing something different.

 

Thanks,

Martin

Re: Force coordinator to pick up updated workflow definition

Rising Star

Also just to confirm - given the behaviour described in the following thread to which I just replied, workflow.xml is not updated in the workspace directory immediately when saved in Hue (as per the response it is only updated when first submitted). So whilst I have made changes to workflow definition (added extra step), the workflow.xml file is not updated in the folder. Is it then expected that the submitted coordinator will pick up those changes?

 

Thanks,

Martin

Re: Force coordinator to pick up updated workflow definition

Rising Star

To follow up: the behaviour I observed is that after the scheduled execution through coordinator, the new actions were shown in the Workflows dashboard Graph tab, however they were skipped during the execution run. See attached screenshot. The new actions introduced were the fork "RunThingsInParallel" and the two actions (both Hive) in the right hand side branch. They are all greyed out rather than the green OK status during and after the execution, which was successful. I will be raising a support call for this.

 

Untitled.png

Re: Force coordinator to pick up updated workflow definition

Rising Star

For the benefit of community members: Cloudera support team confirmed that an updated workflow definition is NOT picked up by an existing submitted coordinator schedule. The workarounds are either to kill the coordinator schedule and resubmit, or to submit the workflow manually which will force the workflow.xml to be updated.

 

Martin

Re: Force coordinator to pick up updated workflow definition

Cloudera Employee
This is JIRA HUE-2551 The two options currently available are - kill the coordinator and create a new one - run the workflow manually after the change.