question Re: How can I WAIT on two processors in NIFI? Final processor needs to check for both before continuing... in Support Questions

How can I WAIT on two processors in NIFI? Final processor needs to check for both before continuing...

jpetro416 — Thu, 16 Mar 2017 02:01:02 GMT

Hi, I'm working on creating a flow like this:

1st processor does work and goes into a funnel

2nd processor does work and routes to same funnel

Final processor needs to check for both before continuing. How do I accomplish this?

NOTE: I do not need the actual file flows, they are only used as trigger files.

Thanks!

Re: How can I WAIT on two processors in NIFI? Final processor needs to check for both before continuing...

bbende — Thu, 16 Mar 2017 03:10:12 GMT

I think you could might be able to use the new Wait and Notify processors that should be in the upcoming 1.2 release of Apache NiFi.

1st processor -> Notify -> Funnel

2nd processor -> Notify -> Funnel

Funnel -> Wait (Signal Count == 2)

Not sure if that is exactly what you described, but seems like it could work.

https://ijokarumawak.github.io/nifi/2017/02/02/nifi-notify-batch/

Re: How can I WAIT on two processors in NIFI? Final processor needs to check for both before continuing...

jpetro416 — Tue, 21 Mar 2017 00:08:16 GMT

Bump for any other possible solutions?

Re: How can I WAIT on two processors in NIFI? Final processor needs to check for both before continuing...

ldaluz — Sun, 18 Aug 2019 12:10:00 GMT

@jpetro416

Another possible solution until the Wait and Notify processors are available.

You can use MergeContent configured to only merge when it hits 2 files (which may work for you because you don't need the final flow files and you just want a trigger in this case):

This first flow trigger will be held in queue before the MergeContent, and will only be merged once the second flow trigger has arrived. The merged flowfile that comes out of the MergeContent will be your final trigger.

Note, I used the following configurations to get this to work:

Back pressure on both 'success' flows going into MergeContent (set to only allow 1 file in queue at a time before the merge)
MergeContent configured with:
- Minimum Number of Entries = 2
- Maximum Number of Entries = 2
- Correlation Attribute Name = 'my-attribute' (I added this attribute to each UpdateAttribute processor - this would only be required if you wanted to have multiple triggers going into the MergeContent and you wanted to control the final trigger based on a correlation attribute)

I've attached the xml flow template for your reference.

mergecontent-trigger-wait.xml

Re: How can I WAIT on two processors in NIFI? Final processor needs to check for both before continuing...

jpetro416 — Wed, 22 Mar 2017 03:20:13 GMT

Thank you, this is exactly the type of workaround I was looking for!