Support Questions

Find answers, ask questions, and share your expertise

delay specific workflow action of oozie coordinator

avatar
Contributor

Hi,

I have to schedule my hadoop jobs with oozie bundle .

1 Oozie bundle (startTime, endTime, kickOffTime) ->1 Oozie Coordinator(startTime, endTime)->2 workflow (task 1, task2).

I need to start task2 workflow after some time ( 45 minutes after) completion of workflow 1.

How can i configure this from coordinator or workflow ?

my bundle :

<?xml version="1.0" encoding="UTF-8"?>

<bundle-app name="ods-jobs-bundle" xmlns="uri:oozie:bundle:0.2">
    <controls>
        <kick-off-time>${kickOffTime}</kick-off-time>
    </controls>
    <coordinator name="ods-ds-cms-coordinator" >
        <app-path>${exampleDir}/ods-ds-cms-coordinator.xml</app-path>
    </coordinator>
</bundle-app>

coordinator :

<coordinator-app name="ods-ds-cms-coordinator" start="${startTime}" end="${endTime}"
                 frequency="60" timezone="${timeZone}" xmlns="uri:oozie:coordinator:0.4">
    <action>
        <workflow>
            <app-path>${exampleDir}/ods-ds-cms-workflow.workflow</app-path>
            <configuration>
                <property>
                    <name>nameNode</name>
                    <value>${nameNode}</value>
                </property>
                <property>
                    <name>jobTracker</name>
                    <value>${jobTracker}</value>
                </property>
                <property>
                    <name>exampleDir</name>
                    <value>${nameNode}/custom/oozie</value>
                </property>
            </configuration>
        </workflow>
    </action>
</coordinator-app>

workflow 1:

<?xml version="1.0" encoding="UTF-8"?>
<workflow-app xmlns="uri:oozie:workflow:0.5" name="ods-ds-cms-workflow.workflow">
  <global>
    <job-tracker>${jobTracker}</job-tracker>
    <name-node>${nameNode}</name-node>
    <configuration>
      <property>
        <name>mapred.job.queue.name</name>
        <value>${queue}</value>
      </property>
    </configuration>
  </global>
  <start to="cms-checker"/>
  <action name="cms-checker">
    <java>
      <main-class>com.insense.helper.CMSPullChecker</main-class>
      <arg>${cmsChecker}</arg>
      <arg>${cmsType}</arg>
      <capture-output/>
    </java>
    <ok to="trigger_next_job"/>
    <error to="kill"/>
  </action>
 

<!-- need to triger this task after 45 minutes of previous task -->
  <action name="trigger_next_job">
    <sub-workflow>
      <app-path>${exampleDir}/ods-next-task.workflow</app-path>
      <propagate-configuration/>
    </sub-workflow>
  <ok to="end"/>
  <error to="kill"/>
  </action>
  <kill name="kill">
    <message>Something went wrong in ods-workflow</message>
  </kill>
  <end name="end"/>
</workflow-app>

workflow 2:

<?xml version="1.0" encoding="UTF-8"?>

<workflow-app xmlns="uri:oozie:workflow:0.5" name="ods-next-task.workflow">
    <global>
        <job-tracker>${jobTracker}</job-tracker>
        <name-node>${nameNode}</name-node>
        <configuration>
            <property>
                <name>mapred.job.queue.name</name>
                <value>${queue}</value>
            </property>
        </configuration>
    </global>
    <start to="campaign-customer-eligibility-mr"/>
    <action name="campaign-customer-eligibility-mr">
        <java>
            <main-class>com.insense.MyMapReduceJobDriver</main-class>
            <capture-output/>
        </java>
        <ok to="done"/>
        <error to="kill"/>
    </action>
    <kill name="kill">
        <message>Something went wrong in ods-workflow</message>
    </kill>
    <end name="done"/>
</workflow-app>

How can I do this from oozie coordinator or workflow .

I can do some thing like java action with Thread.sleep(45 * 60* 1000) but don't wanted to do this .

Thanks

2 REPLIES 2

avatar
Contributor

I think 1 coordinator can only have one workflow as per the xsd for coordinator-

https://oozie.apache.org/docs/4.3.0/CoordinatorFunctionalSpec.html#Oozie_Coordinator_Schema_0.5

<xs:element name="action" type="coordinator:ACTION" minOccurs="1" maxOccurs="1"/>
...

...
<xs:complexType name="ACTION">
        <xs:sequence minOccurs="1" maxOccurs="1">
            <xs:element name="workflow" type="coordinator:WORKFLOW" minOccurs="1" maxOccurs="1"/
            <xs:any namespace="uri:oozie:sla:0.1" minOccurs="0" maxOccurs="1"/>
        </xs:sequence>
    </xs:complexType>

So in description can you clarify what you mean by 1 coordinator with 2 workflows

Bundle jobs take multiple coordinators each with one workflow and you can specify properties for start time/end time for those. Probably you can use those to accomplish the requirement.

avatar
Contributor

Hi @trupti,

There is one coordinator and one workflow and one sub-workflow in side it , I forget to clarify that as my question is how does i start sub-workflow after waiting some time and execution of some actions of main-workflow , using oozie framework help .

Thank You