Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

delay specific workflow action of oozie coordinator

Hi,

I have to schedule my hadoop jobs with oozie bundle .

1 Oozie bundle (startTime, endTime, kickOffTime) ->1 Oozie Coordinator(startTime, endTime)->2 workflow (task 1, task2).

I need to start task2 workflow after some time ( 45 minutes after) completion of workflow 1.

How can i configure this from coordinator or workflow ?

my bundle :

<?xml version="1.0" encoding="UTF-8"?>

<bundle-app name="ods-jobs-bundle" xmlns="uri:oozie:bundle:0.2">
    <controls>
        <kick-off-time>${kickOffTime}</kick-off-time>
    </controls>
    <coordinator name="ods-ds-cms-coordinator" >
        <app-path>${exampleDir}/ods-ds-cms-coordinator.xml</app-path>
    </coordinator>
</bundle-app>

coordinator :

<coordinator-app name="ods-ds-cms-coordinator" start="${startTime}" end="${endTime}"
                 frequency="60" timezone="${timeZone}" xmlns="uri:oozie:coordinator:0.4">
    <action>
        <workflow>
            <app-path>${exampleDir}/ods-ds-cms-workflow.workflow</app-path>
            <configuration>
                <property>
                    <name>nameNode</name>
                    <value>${nameNode}</value>
                </property>
                <property>
                    <name>jobTracker</name>
                    <value>${jobTracker}</value>
                </property>
                <property>
                    <name>exampleDir</name>
                    <value>${nameNode}/custom/oozie</value>
                </property>
            </configuration>
        </workflow>
    </action>
</coordinator-app>

workflow 1:

<?xml version="1.0" encoding="UTF-8"?>
<workflow-app xmlns="uri:oozie:workflow:0.5" name="ods-ds-cms-workflow.workflow">
  <global>
    <job-tracker>${jobTracker}</job-tracker>
    <name-node>${nameNode}</name-node>
    <configuration>
      <property>
        <name>mapred.job.queue.name</name>
        <value>${queue}</value>
      </property>
    </configuration>
  </global>
  <start to="cms-checker"/>
  <action name="cms-checker">
    <java>
      <main-class>com.insense.helper.CMSPullChecker</main-class>
      <arg>${cmsChecker}</arg>
      <arg>${cmsType}</arg>
      <capture-output/>
    </java>
    <ok to="trigger_next_job"/>
    <error to="kill"/>
  </action>
 

<!-- need to triger this task after 45 minutes of previous task -->
  <action name="trigger_next_job">
    <sub-workflow>
      <app-path>${exampleDir}/ods-next-task.workflow</app-path>
      <propagate-configuration/>
    </sub-workflow>
  <ok to="end"/>
  <error to="kill"/>
  </action>
  <kill name="kill">
    <message>Something went wrong in ods-workflow</message>
  </kill>
  <end name="end"/>
</workflow-app>

workflow 2:

<?xml version="1.0" encoding="UTF-8"?>

<workflow-app xmlns="uri:oozie:workflow:0.5" name="ods-next-task.workflow">
    <global>
        <job-tracker>${jobTracker}</job-tracker>
        <name-node>${nameNode}</name-node>
        <configuration>
            <property>
                <name>mapred.job.queue.name</name>
                <value>${queue}</value>
            </property>
        </configuration>
    </global>
    <start to="campaign-customer-eligibility-mr"/>
    <action name="campaign-customer-eligibility-mr">
        <java>
            <main-class>com.insense.MyMapReduceJobDriver</main-class>
            <capture-output/>
        </java>
        <ok to="done"/>
        <error to="kill"/>
    </action>
    <kill name="kill">
        <message>Something went wrong in ods-workflow</message>
    </kill>
    <end name="done"/>
</workflow-app>

How can I do this from oozie coordinator or workflow .

I can do some thing like java action with Thread.sleep(45 * 60* 1000) but don't wanted to do this .

Thanks

2 REPLIES 2

Explorer

I think 1 coordinator can only have one workflow as per the xsd for coordinator-

https://oozie.apache.org/docs/4.3.0/CoordinatorFunctionalSpec.html#Oozie_Coordinator_Schema_0.5

<xs:element name="action" type="coordinator:ACTION" minOccurs="1" maxOccurs="1"/>
...

...
<xs:complexType name="ACTION">
        <xs:sequence minOccurs="1" maxOccurs="1">
            <xs:element name="workflow" type="coordinator:WORKFLOW" minOccurs="1" maxOccurs="1"/
            <xs:any namespace="uri:oozie:sla:0.1" minOccurs="0" maxOccurs="1"/>
        </xs:sequence>
    </xs:complexType>

So in description can you clarify what you mean by 1 coordinator with 2 workflows

Bundle jobs take multiple coordinators each with one workflow and you can specify properties for start time/end time for those. Probably you can use those to accomplish the requirement.

Hi @trupti,

There is one coordinator and one workflow and one sub-workflow in side it , I forget to clarify that as my question is how does i start sub-workflow after waiting some time and execution of some actions of main-workflow , using oozie framework help .

Thank You

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.