NiFi Wait processor, release the amount of flowfiles in the wait queue, but stay closed for incoming flowfiles

[#1]

I'm trying to create a loop within NiFi which gets a lot of files from a multiple TB's FTP-server. Since we don't have enough disk space to save all of these files on the NiFi machine in the provenance history I'm trying to split these up before they are downloaded. I use ListFTP and try to save the 0 byte files in a Wait processor until the flow of a month is finished. When the wait processor gets notified it's opening the gate. But the wait processor isn't fast enough to send all the flowfiles before it gets closed again by the activity.restored -> Notify (close). So flowfiles stay within the Wait processor because it got closed again and didn't go to the next processor yet.

So I either need to know how many flowfiles there are in my wait processor and use it as a variable with the Releasable FlowFile Count variable or the Wait processor must release everything until it's empty and stay close for incoming flowfiles.

How can I achieve this?

As an example I've uploaded the loop I'm trying to create.

[#2]

Hi @jgw.gooijer

Is there any relationship between 'the flow of a month' and the file names of FlowFiles those are listed by FTP and waiting at the Wait processor? If you can derive year and month from a waiting listed file name, by using Attribute Expression Language, then you can configure Wait/Notify processors to use 'year-month' value as signal identifier. This way, you don't have to close identifiers. Keep them open after 'the flow of a month' finished, so that any number of FlowFiles having the same 'year-month' can go through Wait processor.