question Re: Falcon mirror recipe: What happens if an instance is still running when the next is scheduled to start? in Archives of Support Questions (Read Only)

Falcon mirror recipe: What happens if an instance is still running when the next is scheduled to start?

sroberts — Sat, 03 Oct 2015 15:13:19 GMT

When using the Falcon "Mirror Recipe", what happens if an instance is still running when the next is scheduled to start?

Re: Falcon mirror recipe: What happens if an instance is still running when the next is scheduled to start?

bvellanki — Fri, 23 Oct 2015 23:48:35 GMT

The Falcon "Mirror" Recipe has "parallelism" set to 1 since replication needs to run only once to catch up. The next instance will wait for the current instance to complete.

Re: Falcon mirror recipe: What happens if an instance is still running when the next is scheduled to start?

sramesh — Wed, 28 Oct 2015 03:23:07 GMT

If you are asking about using recipes for mirroring, if you look at hdfs-replication-template.xml or hive-replication-template.xml has below properties set to ensure only one instance runs.

<parallel>1</parallel>
<!-- Dir replication needs to run only once to catch up -->     
<order>LAST_ONLY</order>

--------------------------------------------------------------------------------------------------------------

In general, in Falcon this can be controlled using <parallel> and <order> properties in the entity xml. Parallel decides the concurrent replication instances that can run at any given time and order decides the execution order for replication instances like FIFO, LIFO and LAST_ONLY. For replication job needs to run only once to catch up. Setting below configs in entity xml will ensure this:

<parallel>1</parallel>

<order>LAST_ONLY</order>

Parallel maps to concurrency and order maps to execution in Oozie. Please refer Oozie doc for more details.

Re: Falcon mirror recipe: What happens if an instance is still running when the next is scheduled to start?

sroberts — Wed, 28 Oct 2015 05:25:28 GMT

@Sowmya Ramesh - Good answer! Since this was specific to "recipes", can you move the recipe part of the question to the top of your answer and keep the further detail on "parallel" and "order" after that. Then I can accept.

Re: Falcon mirror recipe: What happens if an instance is still running when the next is scheduled to start?

sramesh — Wed, 28 Oct 2015 06:21:48 GMT

@Sean Roberts: Done!

Re: Falcon mirror recipe: What happens if an instance is still running when the next is scheduled to start?

sroberts — Wed, 28 Oct 2015 18:02:04 GMT

Good answer. Somwa's was more complete with a description of how it works.