Created 02-03-2017 12:34 PM
Hi,
I have to schedule my hadoop jobs with oozie bundle .
1 Oozie bundle (startTime, endTime, kickOffTime) ->1 Oozie Coordinator(startTime, endTime)->2 workflow (task 1, task2).
I need to start task2 workflow after some time ( 45 minutes after) completion of workflow 1.
How can i configure this from coordinator or workflow ?
my bundle :
<?xml version="1.0" encoding="UTF-8"?>
<bundle-app name="ods-jobs-bundle" xmlns="uri:oozie:bundle:0.2">
<controls>
<kick-off-time>${kickOffTime}</kick-off-time>
</controls>
<coordinator name="ods-ds-cms-coordinator" >
<app-path>${exampleDir}/ods-ds-cms-coordinator.xml</app-path>
</coordinator>
</bundle-app>coordinator :
<coordinator-app name="ods-ds-cms-coordinator" start="${startTime}" end="${endTime}"
frequency="60" timezone="${timeZone}" xmlns="uri:oozie:coordinator:0.4">
<action>
<workflow>
<app-path>${exampleDir}/ods-ds-cms-workflow.workflow</app-path>
<configuration>
<property>
<name>nameNode</name>
<value>${nameNode}</value>
</property>
<property>
<name>jobTracker</name>
<value>${jobTracker}</value>
</property>
<property>
<name>exampleDir</name>
<value>${nameNode}/custom/oozie</value>
</property>
</configuration>
</workflow>
</action>
</coordinator-app>workflow 1:
<?xml version="1.0" encoding="UTF-8"?>
<workflow-app xmlns="uri:oozie:workflow:0.5" name="ods-ds-cms-workflow.workflow">
<global>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queue}</value>
</property>
</configuration>
</global>
<start to="cms-checker"/>
<action name="cms-checker">
<java>
<main-class>com.insense.helper.CMSPullChecker</main-class>
<arg>${cmsChecker}</arg>
<arg>${cmsType}</arg>
<capture-output/>
</java>
<ok to="trigger_next_job"/>
<error to="kill"/>
</action>
<!-- need to triger this task after 45 minutes of previous task -->
<action name="trigger_next_job">
<sub-workflow>
<app-path>${exampleDir}/ods-next-task.workflow</app-path>
<propagate-configuration/>
</sub-workflow>
<ok to="end"/>
<error to="kill"/>
</action>
<kill name="kill">
<message>Something went wrong in ods-workflow</message>
</kill>
<end name="end"/>
</workflow-app>workflow 2:
<?xml version="1.0" encoding="UTF-8"?>
<workflow-app xmlns="uri:oozie:workflow:0.5" name="ods-next-task.workflow">
<global>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queue}</value>
</property>
</configuration>
</global>
<start to="campaign-customer-eligibility-mr"/>
<action name="campaign-customer-eligibility-mr">
<java>
<main-class>com.insense.MyMapReduceJobDriver</main-class>
<capture-output/>
</java>
<ok to="done"/>
<error to="kill"/>
</action>
<kill name="kill">
<message>Something went wrong in ods-workflow</message>
</kill>
<end name="done"/>
</workflow-app>How can I do this from oozie coordinator or workflow .
I can do some thing like java action with Thread.sleep(45 * 60* 1000) but don't wanted to do this .
Thanks
Created 02-07-2017 12:15 AM
I think 1 coordinator can only have one workflow as per the xsd for coordinator-
https://oozie.apache.org/docs/4.3.0/CoordinatorFunctionalSpec.html#Oozie_Coordinator_Schema_0.5
<xs:element name="action" type="coordinator:ACTION" minOccurs="1" maxOccurs="1"/>
...
...
<xs:complexType name="ACTION">
<xs:sequence minOccurs="1" maxOccurs="1">
<xs:element name="workflow" type="coordinator:WORKFLOW" minOccurs="1" maxOccurs="1"/
<xs:any namespace="uri:oozie:sla:0.1" minOccurs="0" maxOccurs="1"/>
</xs:sequence>
</xs:complexType>
So in description can you clarify what you mean by 1 coordinator with 2 workflows
Bundle jobs take multiple coordinators each with one workflow and you can specify properties for start time/end time for those. Probably you can use those to accomplish the requirement.
Created 02-07-2017 05:26 AM
Hi @trupti,
There is one coordinator and one workflow and one sub-workflow in side it , I forget to clarify that as my question is how does i start sub-workflow after waiting some time and execution of some actions of main-workflow , using oozie framework help .
Thank You