Reset of Target Signal Count in Wait Processor

[#1]

I have a flow in which I wait for 5 files to come out from each of the respective executeProcessors, after which I start another processor. I achieved this using Wait and Notify processors by target signal count given as 5. I stop and restart the flow, it works as expected. Now, my question is when and how would this counter be reset? If the executeProcessors are scheduled to run for every 30 min, do signals from notify keep increasing the counter. If so, how does my wait processor ever match the signal count of 5 that I gave and proceed to next processor every time.

Thanks,

John

[#2]

@John Carter

What does the configuration of the Wait processor look like?

[#3]

Hi Wyner,

wait-processor.jpeg

You can see that am waiting for 10 signals here.

[#4]

You'd probably want to have a signal identifier using a unique flow file attribute (such as uuid). If you could provide more details around what you want to achieve, that'd be helpful.

[#5]

I am using file name as my signal identifier. Filenames before notify and wait are all renamed to a specific value and the count of these signals is what I am checking.

[#6 · Accepted solution]

@John Carter

Assuming you have the flow designed correctly.

You don't have to reset the counter. That is why the Wait/Notify use the DistributedMapCacheClientService. It will keep track of the signal count.

Would you be willing to provide a snapshot of your flow? Also, a snapshot of how the DistributedMapCacheServer controller service is configured.

[#7]

@John Carter

Are you not seeing the behavior that you expect? You don't need to stop and start the flow, that has no effect on the count of the flow files the Wait processor is waiting for.

Would you be willing to provide a snapshot of your flow?

[#8]

Thanks for your reply, Wyner. Here is the screen shot of the flow that I simulated from the actual ones that we have. Also, the configurations of wait and notify that I am using are shown here:

1. Generate flow file generates a flow file every 5 seconds

2. Flow files are renamed to 'release_signal' (as this is what I am using as release signal identifier in wait and notify processors)

3. When there are 5 such signals, I want wait processor to push all the 5 flow files to the downstream success relation. Schedule on wait processor is 10 sec

Given the above scenario, I expect the success relation from wait processor should get flow files in the steps 5.. (like 5,10,15 ..) But I do not see something like that. You may see it in the screen shot too. There are 13 files in total that are pushed to success relation from wait. What I observed is, sometimes 5 flow files are pushed at once. But sometimes even one flow file is pushed to success from wait which I don't understand why that happens.

Since I could not get this working as expected, I switched to using "Merge Content" and it works for the use case that I have at hand.

Flow:

Update Attribute:

Notify:

Wait:

[#9]

@John Carter

I am using a similar setup for my testing and it works just like you think it should.

For your test, make sure and clear any residual file counts, by stopping and starting the DistributedMapCacheServer and the DistributedMapCacheClientService controller services. Then restart the test.

UpdateAttribute

Notify

Wait

Required controller services

[#10]

@John Carter

Did you get chance to look at my flow? Also, will you post a snapshot of your DistributedMapCacheClientService configuration? Your flow should work like you expect.