Member since
08-28-2017
2
Posts
0
Kudos Received
0
Solutions
08-28-2017
03:14 PM
I think I have found the issue--I have incorrectly specified the date for the dataset "instance" as ${start_date} when it should be ${coord:current(0)}. I suspect that by replacing: <input-events>
<data-in name="datatype_input_event_trigger" dataset="trigger_dataset_name">
<instance>${start_date}</instance>
</data-in>
</input-events>
...with... <input-events>
<data-in name="datatype_input_event_trigger" dataset="trigger_dataset_name">
<instance>${coord:current(0)}</instance>
</data-in>
</input-events> my issue should be resolved.
... View more
08-28-2017
02:57 PM
Everything that I have read suggests that ${YEAR}, ${MONTH}, and ${DAY} can be used to specify the uri-template for date-based Oozie coordinator datasets (see, e.g., [1], [2]). However, after starting my coordinator via a bundle (see below) on '2017-08-26', Oozie continues to look in the directory '2017-08/26' for the presence of my trigger file (which has existed since the 26th). A StackOverflow post [3] was able to resolve a similar issue by replacing ${YEAR} with ${coord:formatTime(coord:nominalTime(),'yyyy')}, but this seems to break with the simplicity of using Oozie's EL Time Constants.
Am I missing something regarding the proper usage of ${YEAR}, ${MONTH}, and ${DAY} for my coordinator's uri-template? Thanks! - Matt Links: [1] Oozie - . (2017). Oozie.apache.org. Retrieved 28 August 2017, from https://oozie.apache.org/docs/4.3.0/CoordinatorFunctionalSpec.html#a5.1._Synchronous_Datasets [2] Wu, Jui-Chung. (2017). GitHub: YahooArchive/oozie. Retrieved 28 August 2017, from https://github.com/YahooArchive/oozie/wiki/Oozie-Coord-Use-Cases [3] spijs. (2017). How to set current month, day or year as workflow parameter in a coordinator (on Hue). Stackoverflow.com. Retrieved 28 August 2017, from https://stackoverflow.com/questions/41744794/how-to-set-current-month-day-or-year-as-workflow-parameter-in-a-coordinator-on Oozie Version: Oozie server build version: 4.2.0.2.5.0.0-1245 Bundle Source: <bundle-app
xmlns="uri:oozie:bundle:0.2" name="run_all_my_coordinators">
<controls>
<kick-off-time>${start_date}</kick-off-time>
</controls>
<coordinator name="notify_oozie_start">
<app-path>${oozieProjectRoot}/notify_oozie_start</app-path>
<configuration>
<property>
<name>datatype_name</name>
<value>${my_bundle_datatype}</value>
</property>
</configuration>
</coordinator>
...
<coordinator name="my_coordinator">
<app-path>${coord_path}</app-path>
<configuration>
<property>
<name>datatype_name</name>
<value>${my_datatype_name}</value>
</property>
<property>
<name>datatype_dir</name>
<value>${my_datatype_dir}</value>
</property>
</configuration>
</coordinator>
...
<coordinator name="notify_oozie_end">
<app-path>${oozieProjectRoot}/notify_oozie_end</app-path>
<configuration>
<property>
<name>datatype_name</name>
<value>${my_bundle_datatype}</value>
</property>
</configuration>
</coordinator>
</bundle-app>
Coordinator Source: <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<coordinator-app xmlns="uri:oozie:coordinator:0.5"
name="my_coordinator"
start="${start_date}"
end="${end_date}"
timezone="GMT-05:00"
frequency="${coord:days(1)}">
<controls>
<concurrency>1</concurrency>
</controls>
<datasets>
<dataset frequency="${coord:days(1)}"
initial-instance="${initial_instance}"
name="trigger_dataset_name"
timezone="GMT-05:00">
<uri-template>
${trigger_uri_template}/${YEAR}-${MONTH}/${DAY}
</uri-template>
<done-flag>${datatype_name_etl}${start_flag_suffix}</done-flag>
</dataset>
</datasets>
<input-events>
<data-in name="datatype_input_event_trigger" dataset="trigger_dataset_name">
<instance>${start_date}</instance>
</data-in>
</input-events>
<action>
<workflow>
<app-path>${workflow_path}</app-path>
<configuration>
<property>
<name>year</name>
<value>${coord:formatTime(coord:nominalTime(), 'yyyy')}</value>
</property>
<property>
<name>month</name>
<value>${coord:formatTime(coord:nominalTime(), 'MM')}</value>
</property>
<property>
<name>day</name>
<value>${coord:formatTime(coord:nominalTime(), 'dd')}</value>
</property>
</configuration>
</workflow>
</action>
</coordinator-app>
... View more
Labels:
- Labels:
-
Apache Oozie