Support Questions
Find answers, ask questions, and share your expertise

Apache Nifi - use of Notify and Wait with unknown amount of files

Solved Go to solution
Highlighted

Apache Nifi - use of Notify and Wait with unknown amount of files

I tried to find out on myself, but after days of trying I decided that I need help. My usecase is to concat splitted zip-files (FilaA.zip.001, 002, 003 - 00x) and  I don't really know how many files will be delivered.  The files  are big and delivery takes time... The last file  to be delivered is a notification-file "ready.txt"

 

My current workflow is like this:

- receive alle files

-extract   number of zip-file   (001-x) and build a priority (999-number of zip-file) of it

-move to a queue with a PriorityAttributePrioritizer

-extract max number of files (i.e. 7 from 007 - the file with priority 992)

-move files via  notify processer to the wait processor

MelbarKasom_0-1589556094961.png

- in the wait processor I want to wait until the notfication-file  arrived and  deliver all the files in wait-queue

MelbarKasom_1-1589556254435.png

-after the the files will be concatenated  with the MergeContent processor and unzipped.

 

My configuration works only one time.  ... The second delivery with exact the same filenames as in the first delivery is directly moved to "success" and not into the wait queue... 

What I am doing wrong ?

 

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Apache Nifi - use of Notify and Wait with unknown amount of files

I created a new MapCacheService and changed the release condition back to 

MelbarKasom_0-1589557626903.png

now it seems to work

View solution in original post

2 REPLIES 2
Highlighted

Re: Apache Nifi - use of Notify and Wait with unknown amount of files

I created a new MapCacheService and changed the release condition back to 

MelbarKasom_0-1589557626903.png

now it seems to work

View solution in original post

Re: Apache Nifi - use of Notify and Wait with unknown amount of files

Community Manager

I'm happy to see you resolved your issue. Please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. 

Screen Shot 2019-08-06 at 1.54.47 PM.png

 

 


Cy Jervis, Community Manager

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:
Community Guidelines
How to use the forum
Don't have an account?