question Re: Reset of Target Signal Count in Wait Processor in Support Questions

Reset of Target Signal Count in Wait Processor

johncarteratfl — Sat, 16 Sep 2017 02:33:28 GMT

I have a flow in which I wait for 5 files to come out from each of the respective executeProcessors, after which I start another processor. I achieved this using Wait and Notify processors by target signal count given as 5. I stop and restart the flow, it works as expected. Now, my question is when and how would this counter be reset? If the executeProcessors are scheduled to run for every 30 min, do signals from notify keep increasing the counter. If so, how does my wait processor ever match the signal count of 5 that I gave and proceed to next processor every time.

Thanks,

John

Re: Reset of Target Signal Count in Wait Processor

Wynner — Tue, 19 Sep 2017 03:16:33 GMT

@John Carter

What does the configuration of the Wait processor look like?

Re: Reset of Target Signal Count in Wait Processor

johncarteratfl — Tue, 19 Sep 2017 04:46:07 GMT

Hi Wyner,

wait-processor.jpeg

You can see that am waiting for 10 signals here.

Re: Reset of Target Signal Count in Wait Processor

pvillard — Tue, 19 Sep 2017 16:37:10 GMT

You'd probably want to have a signal identifier using a unique flow file attribute (such as uuid). If you could provide more details around what you want to achieve, that'd be helpful.

Re: Reset of Target Signal Count in Wait Processor

Wynner — Tue, 19 Sep 2017 19:52:11 GMT

@John Carter

Assuming you have the flow designed correctly.

You don't have to reset the counter. That is why the Wait/Notify use the DistributedMapCacheClientService. It will keep track of the signal count.

Would you be willing to provide a snapshot of your flow? Also, a snapshot of how the DistributedMapCacheServer controller service is configured.

Re: Reset of Target Signal Count in Wait Processor

Wynner — Mon, 25 Sep 2017 03:04:00 GMT

@John Carter

Are you not seeing the behavior that you expect? You don't need to stop and start the flow, that has no effect on the count of the flow files the Wait processor is waiting for.

Would you be willing to provide a snapshot of your flow?

Re: Reset of Target Signal Count in Wait Processor

johncarteratfl — Sun, 18 Aug 2019 06:30:41 GMT

Thanks for your reply, Wyner. Here is the screen shot of the flow that I simulated from the actual ones that we have. Also, the configurations of wait and notify that I am using are shown here:

1. Generate flow file generates a flow file every 5 seconds

2. Flow files are renamed to 'release_signal' (as this is what I am using as release signal identifier in wait and notify processors)

3. When there are 5 such signals, I want wait processor to push all the 5 flow files to the downstream success relation. Schedule on wait processor is 10 sec

Given the above scenario, I expect the success relation from wait processor should get flow files in the steps 5.. (like 5,10,15 ..) But I do not see something like that. You may see it in the screen shot too. There are 13 files in total that are pushed to success relation from wait. What I observed is, sometimes 5 flow files are pushed at once. But sometimes even one flow file is pushed to success from wait which I don't understand why that happens.

Since I could not get this working as expected, I switched to using "Merge Content" and it works for the use case that I have at hand.

Flow:

Update Attribute:

Notify:

Wait:

Re: Reset of Target Signal Count in Wait Processor

johncarteratfl — Wed, 27 Sep 2017 01:40:05 GMT

I am using file name as my signal identifier. Filenames before notify and wait are all renamed to a specific value and the count of these signals is what I am checking.

Re: Reset of Target Signal Count in Wait Processor

Wynner — Sun, 18 Aug 2019 06:30:17 GMT

@John Carter

I am using a similar setup for my testing and it works just like you think it should.

For your test, make sure and clear any residual file counts, by stopping and starting the DistributedMapCacheServer and the DistributedMapCacheClientService controller services. Then restart the test.

UpdateAttribute

Notify

Wait

Required controller services

Re: Reset of Target Signal Count in Wait Processor

Wynner — Thu, 28 Sep 2017 21:57:32 GMT

@John Carter

Did you get chance to look at my flow? Also, will you post a snapshot of your DistributedMapCacheClientService configuration? Your flow should work like you expect.

Re: Reset of Target Signal Count in Wait Processor

johncarteratfl — Mon, 02 Oct 2017 21:55:56 GMT

Hi Wyner, Yes, after i disabled and enabled the MapCahce service configuration, it was working fine. Thanks for that.

Re: Reset of Target Signal Count in Wait Processor

Wynner — Tue, 03 Oct 2017 02:12:28 GMT

@John Carter

Would you accept the answer so that others know it resolved your issue?