Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

delay specific workflow action of oozie coordinator

Highlighted

delay specific workflow action of oozie coordinator

New Contributor

Hi,

I have to schedule my hadoop jobs with oozie bundle .

1 Oozie bundle (startTime, endTime, kickOffTime) ->1 Oozie Coordinator(startTime, endTime)->2 workflow (task 1, task2).

I need to start task2 workflow after some time ( 45 minutes after) completion of workflow 1.

How can i configure this from coordinator or workflow ?

my bundle :

<?xml version="1.0" encoding="UTF-8"?>

<bundle-app name="ods-jobs-bundle" xmlns="uri:oozie:bundle:0.2">
    <controls>
        <kick-off-time>${kickOffTime}</kick-off-time>
    </controls>
    <coordinator name="ods-ds-cms-coordinator" >
        <app-path>${exampleDir}/ods-ds-cms-coordinator.xml</app-path>
    </coordinator>
</bundle-app>

coordinator :

<coordinator-app name="ods-ds-cms-coordinator" start="${startTime}" end="${endTime}"
                 frequency="60" timezone="${timeZone}" xmlns="uri:oozie:coordinator:0.4">
    <action>
        <workflow>
            <app-path>${exampleDir}/ods-ds-cms-workflow.workflow</app-path>
            <configuration>
                <property>
                    <name>nameNode</name>
                    <value>${nameNode}</value>
                </property>
                <property>
                    <name>jobTracker</name>
                    <value>${jobTracker}</value>
                </property>
                <property>
                    <name>exampleDir</name>
                    <value>${nameNode}/custom/oozie</value>
                </property>
            </configuration>
        </workflow>
    </action>
</coordinator-app>

workflow 1:

<?xml version="1.0" encoding="UTF-8"?>
<workflow-app xmlns="uri:oozie:workflow:0.5" name="ods-ds-cms-workflow.workflow">
  <global>
    <job-tracker>${jobTracker}</job-tracker>
    <name-node>${nameNode}</name-node>
    <configuration>
      <property>
        <name>mapred.job.queue.name</name>
        <value>${queue}</value>
      </property>
    </configuration>
  </global>
  <start to="cms-checker"/>
  <action name="cms-checker">
    <java>
      <main-class>com.insense.helper.CMSPullChecker</main-class>
      <arg>${cmsChecker}</arg>
      <arg>${cmsType}</arg>
      <capture-output/>
    </java>
    <ok to="trigger_next_job"/>
    <error to="kill"/>
  </action>
 

<!-- need to triger this task after 45 minutes of previous task -->
  <action name="trigger_next_job">
    <sub-workflow>
      <app-path>${exampleDir}/ods-next-task.workflow</app-path>
      <propagate-configuration/>
    </sub-workflow>
  <ok to="end"/>
  <error to="kill"/>
  </action>
  <kill name="kill">
    <message>Something went wrong in ods-workflow</message>
  </kill>
  <end name="end"/>
</workflow-app>

workflow 2:

<?xml version="1.0" encoding="UTF-8"?>

<workflow-app xmlns="uri:oozie:workflow:0.5" name="ods-next-task.workflow">
    <global>
        <job-tracker>${jobTracker}</job-tracker>
        <name-node>${nameNode}</name-node>
        <configuration>
            <property>
                <name>mapred.job.queue.name</name>
                <value>${queue}</value>
            </property>
        </configuration>
    </global>
    <start to="campaign-customer-eligibility-mr"/>
    <action name="campaign-customer-eligibility-mr">
        <java>
            <main-class>com.insense.MyMapReduceJobDriver</main-class>
            <capture-output/>
        </java>
        <ok to="done"/>
        <error to="kill"/>
    </action>
    <kill name="kill">
        <message>Something went wrong in ods-workflow</message>
    </kill>
    <end name="done"/>
</workflow-app>

How can I do this from oozie coordinator or workflow .

I can do some thing like java action with Thread.sleep(45 * 60* 1000) but don't wanted to do this .

Thanks

2 REPLIES 2

Re: delay specific workflow action of oozie coordinator

New Contributor

I think 1 coordinator can only have one workflow as per the xsd for coordinator-

https://oozie.apache.org/docs/4.3.0/CoordinatorFunctionalSpec.html#Oozie_Coordinator_Schema_0.5

<xs:element name="action" type="coordinator:ACTION" minOccurs="1" maxOccurs="1"/>
...

...
<xs:complexType name="ACTION">
        <xs:sequence minOccurs="1" maxOccurs="1">
            <xs:element name="workflow" type="coordinator:WORKFLOW" minOccurs="1" maxOccurs="1"/
            <xs:any namespace="uri:oozie:sla:0.1" minOccurs="0" maxOccurs="1"/>
        </xs:sequence>
    </xs:complexType>

So in description can you clarify what you mean by 1 coordinator with 2 workflows

Bundle jobs take multiple coordinators each with one workflow and you can specify properties for start time/end time for those. Probably you can use those to accomplish the requirement.

Re: delay specific workflow action of oozie coordinator

New Contributor

Hi @trupti,

There is one coordinator and one workflow and one sub-workflow in side it , I forget to clarify that as my question is how does i start sub-workflow after waiting some time and execution of some actions of main-workflow , using oozie framework help .

Thank You