Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to configure Oozie coordinator dataset for previous day with path pattern as BasePath/YYYYMMdd/00/_Complete

Highlighted

How to configure Oozie coordinator dataset for previous day with path pattern as BasePath/YYYYMMdd/00/_Complete

New Contributor
0favorite

I want to run workflow based on availability of Control files for previous date. Date format in my directory is ${basePath}/YYYYMMdd/00/_Complete.I want to check the _Complete file inside /00/. My Job will run daily on the previous day data i.e. on ${basePath}/20160817/00/. I tried the options provided in similar questions But still not working. When I am testing it for same day data with below value for <instance> , it is working But Not with coord:current(-1) option, It still checks for same day date

<instance>${coord:current(0)}</instance>

Is there any restriction on URI-TEMPLATE formats, meaning do we need to have it in fixed format basepath/${YEAR}${$MONTH}${DAY}/Complete and Not like basepath/${YEAR}${$MONTH}${DAY}/00/Complete. Please help.

Here is the dryrun output for my Coordinator job.

***coordJob after parsing: ***
<coordinator-app xmlns="uri:oozie:coordinator:0.1" name="my_Scheduler_5f" frequency="1" start="2016-08-17T23:40Z" end="2016-08-19T23:45Z" timezone="America/Los_Angeles" freq_timeunit="DAY" end_of_duration="NONE">
  <controls>
    <timeout>30</timeout>
  </controls>
  <input-events>
    <data-in name="coordInput_1" dataset="input1">
      <dataset name="input1" frequency="1" initial-instance="2016-08-17T00:00Z" timezone="America/Los_Angeles" freq_timeunit="DAY" end_of_duration="NONE">
        <uri-template>${nameNode}/myHdfsPath/Finalpath1/${YEAR}${MONTH}${DAY}/00/</uri-template>
        <done-flag>_Complete</done-flag>
      </dataset>
      <instance>${coord:current(-1)}</instance>
    </data-in>
    <data-in name="coordInput_2" dataset="input2">
      <dataset name="input2" frequency="1" initial-instance="2016-08-17T23:00Z" timezone="America/Los_Angeles" freq_timeunit="DAY" end_of_duration="NONE">
        <uri-template>${nameNode}/myHdfsPath/Finalpath2/${YEAR}${MONTH}${DAY}/00/</uri-template>
        <done-flag>_Complete</done-flag>
      </dataset>
      <instance>${coord:current(-1)}</instance>
    </data-in>
  </input-events>
  <action>
    <workflow>
      <app-path>${nameNode}/myHdfsPath/My_POC/wf-app-dir</app-path>
      <configuration>
        <property>
          <name>date</name>
          <value>${coord:formatTime(coord:dateOffset(coord:actualTime(),-1,'DAY'), "yyyyMMdd")}</value>
        </property>
    </workflow>
  </action>
</coordinator-app>
***actions for instance***
Don't have an account?
Coming from Hortonworks? Activate your account here