Created 02-03-2017 12:34 PM
Hi,
I have to schedule my hadoop jobs with oozie bundle .
1 Oozie bundle (startTime, endTime, kickOffTime) ->1 Oozie Coordinator(startTime, endTime)->2 workflow (task 1, task2).
I need to start task2 workflow after some time ( 45 minutes after) completion of workflow 1.
How can i configure this from coordinator or workflow ?
my bundle :
<?xml version="1.0" encoding="UTF-8"?> <bundle-app name="ods-jobs-bundle" xmlns="uri:oozie:bundle:0.2"> <controls> <kick-off-time>${kickOffTime}</kick-off-time> </controls> <coordinator name="ods-ds-cms-coordinator" > <app-path>${exampleDir}/ods-ds-cms-coordinator.xml</app-path> </coordinator> </bundle-app>
coordinator :
<coordinator-app name="ods-ds-cms-coordinator" start="${startTime}" end="${endTime}" frequency="60" timezone="${timeZone}" xmlns="uri:oozie:coordinator:0.4"> <action> <workflow> <app-path>${exampleDir}/ods-ds-cms-workflow.workflow</app-path> <configuration> <property> <name>nameNode</name> <value>${nameNode}</value> </property> <property> <name>jobTracker</name> <value>${jobTracker}</value> </property> <property> <name>exampleDir</name> <value>${nameNode}/custom/oozie</value> </property> </configuration> </workflow> </action> </coordinator-app>
workflow 1:
<?xml version="1.0" encoding="UTF-8"?> <workflow-app xmlns="uri:oozie:workflow:0.5" name="ods-ds-cms-workflow.workflow"> <global> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <configuration> <property> <name>mapred.job.queue.name</name> <value>${queue}</value> </property> </configuration> </global> <start to="cms-checker"/> <action name="cms-checker"> <java> <main-class>com.insense.helper.CMSPullChecker</main-class> <arg>${cmsChecker}</arg> <arg>${cmsType}</arg> <capture-output/> </java> <ok to="trigger_next_job"/> <error to="kill"/> </action> <!-- need to triger this task after 45 minutes of previous task --> <action name="trigger_next_job"> <sub-workflow> <app-path>${exampleDir}/ods-next-task.workflow</app-path> <propagate-configuration/> </sub-workflow> <ok to="end"/> <error to="kill"/> </action> <kill name="kill"> <message>Something went wrong in ods-workflow</message> </kill> <end name="end"/> </workflow-app>
workflow 2:
<?xml version="1.0" encoding="UTF-8"?> <workflow-app xmlns="uri:oozie:workflow:0.5" name="ods-next-task.workflow"> <global> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <configuration> <property> <name>mapred.job.queue.name</name> <value>${queue}</value> </property> </configuration> </global> <start to="campaign-customer-eligibility-mr"/> <action name="campaign-customer-eligibility-mr"> <java> <main-class>com.insense.MyMapReduceJobDriver</main-class> <capture-output/> </java> <ok to="done"/> <error to="kill"/> </action> <kill name="kill"> <message>Something went wrong in ods-workflow</message> </kill> <end name="done"/> </workflow-app>
How can I do this from oozie coordinator or workflow .
I can do some thing like java action with Thread.sleep(45 * 60* 1000) but don't wanted to do this .
Thanks
Created 02-07-2017 12:15 AM
I think 1 coordinator can only have one workflow as per the xsd for coordinator-
https://oozie.apache.org/docs/4.3.0/CoordinatorFunctionalSpec.html#Oozie_Coordinator_Schema_0.5
<xs:element name="action" type="coordinator:ACTION" minOccurs="1" maxOccurs="1"/> ... ... <xs:complexType name="ACTION"> <xs:sequence minOccurs="1" maxOccurs="1"> <xs:element name="workflow" type="coordinator:WORKFLOW" minOccurs="1" maxOccurs="1"/ <xs:any namespace="uri:oozie:sla:0.1" minOccurs="0" maxOccurs="1"/> </xs:sequence> </xs:complexType>
So in description can you clarify what you mean by 1 coordinator with 2 workflows
Bundle jobs take multiple coordinators each with one workflow and you can specify properties for start time/end time for those. Probably you can use those to accomplish the requirement.
Created 02-07-2017 05:26 AM
Hi @trupti,
There is one coordinator and one workflow and one sub-workflow in side it , I forget to clarify that as my question is how does i start sub-workflow after waiting some time and execution of some actions of main-workflow , using oozie framework help .
Thank You