Falcon mirror recipe: What happens if an instance is still running when the next is scheduled to start?

[#1]

When using the Falcon "Mirror Recipe", what happens if an instance is still running when the next is scheduled to start?

[#2]

The Falcon "Mirror" Recipe has "parallelism" set to 1 since replication needs to run only once to catch up. The next instance will wait for the current instance to complete.

[#3]

Good answer. Somwa's was more complete with a description of how it works.

[#4 · Accepted solution]

If you are asking about using recipes for mirroring, if you look at hdfs-replication-template.xml or hive-replication-template.xml has below properties set to ensure only one instance runs.

<parallel>1</parallel>
<!-- Dir replication needs to run only once to catch up -->     
<order>LAST_ONLY</order>

--------------------------------------------------------------------------------------------------------------

In general, in Falcon this can be controlled using <parallel> and <order> properties in the entity xml. Parallel decides the concurrent replication instances that can run at any given time and order decides the execution order for replication instances like FIFO, LIFO and LAST_ONLY. For replication job needs to run only once to catch up. Setting below configs in entity xml will ensure this:

<parallel>1</parallel>

<order>LAST_ONLY</order>

Parallel maps to concurrency and order maps to execution in Oozie. Please refer Oozie doc for more details.

[#5]

@Sowmya Ramesh - Good answer! Since this was specific to "recipes", can you move the recipe part of the question to the top of your answer and keep the further detail on "parallel" and "order" after that. Then I can accept.

[#6]

@Sean Roberts: Done!