Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Reset of Target Signal Count in Wait Processor

avatar

I have a flow in which I wait for 5 files to come out from each of the respective executeProcessors, after which I start another processor. I achieved this using Wait and Notify processors by target signal count given as 5. I stop and restart the flow, it works as expected. Now, my question is when and how would this counter be reset? If the executeProcessors are scheduled to run for every 30 min, do signals from notify keep increasing the counter. If so, how does my wait processor ever match the signal count of 5 that I gave and proceed to next processor every time.

Thanks,

John

1 ACCEPTED SOLUTION

avatar

@John Carter

Assuming you have the flow designed correctly.

You don't have to reset the counter. That is why the Wait/Notify use the DistributedMapCacheClientService. It will keep track of the signal count.

Would you be willing to provide a snapshot of your flow? Also, a snapshot of how the DistributedMapCacheServer controller service is configured.

View solution in original post

11 REPLIES 11

avatar
@John Carter

What does the configuration of the Wait processor look like?

avatar

Hi Wyner,

wait-processor.jpeg

You can see that am waiting for 10 signals here.

avatar

You'd probably want to have a signal identifier using a unique flow file attribute (such as uuid). If you could provide more details around what you want to achieve, that'd be helpful.

avatar

I am using file name as my signal identifier. Filenames before notify and wait are all renamed to a specific value and the count of these signals is what I am checking.

avatar

@John Carter

Assuming you have the flow designed correctly.

You don't have to reset the counter. That is why the Wait/Notify use the DistributedMapCacheClientService. It will keep track of the signal count.

Would you be willing to provide a snapshot of your flow? Also, a snapshot of how the DistributedMapCacheServer controller service is configured.

avatar
@John Carter

Are you not seeing the behavior that you expect? You don't need to stop and start the flow, that has no effect on the count of the flow files the Wait processor is waiting for.

Would you be willing to provide a snapshot of your flow?

avatar

Thanks for your reply, Wyner. Here is the screen shot of the flow that I simulated from the actual ones that we have. Also, the configurations of wait and notify that I am using are shown here:

1. Generate flow file generates a flow file every 5 seconds

2. Flow files are renamed to 'release_signal' (as this is what I am using as release signal identifier in wait and notify processors)

3. When there are 5 such signals, I want wait processor to push all the 5 flow files to the downstream success relation. Schedule on wait processor is 10 sec

Given the above scenario, I expect the success relation from wait processor should get flow files in the steps 5.. (like 5,10,15 ..) But I do not see something like that. You may see it in the screen shot too. There are 13 files in total that are pushed to success relation from wait. What I observed is, sometimes 5 flow files are pushed at once. But sometimes even one flow file is pushed to success from wait which I don't understand why that happens.

Since I could not get this working as expected, I switched to using "Merge Content" and it works for the use case that I have at hand.

Flow:

39554-pic1.jpeg

Update Attribute:

39555-update-attr.jpeg

Notify:

39556-notify.jpeg

Wait:

39557-wait.jpeg

avatar
@John Carter

I am using a similar setup for my testing and it works just like you think it should.

For your test, make sure and clear any residual file counts, by stopping and starting the DistributedMapCacheServer and the DistributedMapCacheClientService controller services. Then restart the test.

39558-screen-shot-2017-09-26-at-30519-pm.png

UpdateAttribute

39559-screen-shot-2017-09-26-at-30727-pm.png

Notify

39560-screen-shot-2017-09-26-at-30905-pm.png

Wait

39561-screen-shot-2017-09-26-at-30939-pm.png

Required controller services

39562-screen-shot-2017-09-26-at-31032-pm.png

avatar
@John Carter

Did you get chance to look at my flow? Also, will you post a snapshot of your DistributedMapCacheClientService configuration? Your flow should work like you expect.