Support Questions
Find answers, ask questions, and share your expertise

oozie datasets dependency not very synchronized

oozie datasets dependency not very synchronized

Hi,

I noticed something curious about datasets (used in output-events).

Considering a coordinator "A" (the dependence) with these parameters :
frequency="${coord:minutes(5)}
dataset / frequency : ${coord:hours(1)}
dataset / uri-template : /temp/${YEAR}/${MONTH}/${DAY}/${HOUR}

Considering a coordinator "Z" waiting for the dependencies and executing itself every 5 minutes, started at 2:18 for example...

I have this when I look the coordinator "Z":

nominal time | missing dependencies  |last modified time
-------------+-----------------------+------------------
8:10         | /temp/2017/11/08/07   | 7:18
8:15         | /temp/2017/11/08/07   | 7:18
8:20         | /temp/2017/11/08/08   | 8:18

What I understand is :
The modification time implies the dependency evaluation
Before 8:18, it is still the 2017/11/08/07 file to be needed, because the previous modification is at 7 (7:18 precisely)

It is the normal behaviour ? Or am I doing something weird somewhere :) ?

Thanks.