Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

oozie datasets dependency not very synchronized

Highlighted

oozie datasets dependency not very synchronized

New Contributor

Hi,

I noticed something curious about datasets (used in output-events).

Considering a coordinator "A" (the dependence) with these parameters :
frequency="${coord:minutes(5)}
dataset / frequency : ${coord:hours(1)}
dataset / uri-template : /temp/${YEAR}/${MONTH}/${DAY}/${HOUR}

Considering a coordinator "Z" waiting for the dependencies and executing itself every 5 minutes, started at 2:18 for example...

I have this when I look the coordinator "Z":

nominal time | missing dependencies  |last modified time
-------------+-----------------------+------------------
8:10         | /temp/2017/11/08/07   | 7:18
8:15         | /temp/2017/11/08/07   | 7:18
8:20         | /temp/2017/11/08/08   | 8:18

What I understand is :
The modification time implies the dependency evaluation
Before 8:18, it is still the 2017/11/08/07 file to be needed, because the previous modification is at 7 (7:18 precisely)

It is the normal behaviour ? Or am I doing something weird somewhere :) ?

Thanks.