In NiFi how to write flowfiles into HDFS only if all mandatory files are received?

[#1]

Hi,

I am expecting two files, file with abc.txt and file with pqr.txt my next processor should get triggered only if these two files are received.

Currently i am using listFile fetchFIle to capture the changes of source directory. Once i have received two mandatory files my next process is to process them. Now i am not sure how should i configure NiFi to hold the file in queue until i got all mandatory files.

Do we know how should we achieve this using NiFi?

[#2]

@RAUI

-

You may want to look in to using the "Wait" and "Notify" processors to accomplish this. The Wait processor would route a FlowFile to the wait relationship (default configuration) until a release signal is written to the cache service by the notify processor. So you route one file to wait and the other to Notify.

-

Do you know what order the files will always arrive in? If files arrive in no specific order, you may need two wait/notify sets of processors. One flow path has a wait feeding a notify and the other a notify feeding a wait. That will result in a wait occurring on either FlowFile no matter which file is received first.

-

Thanks,

Matt

-

When an "Answer" addresses/solves your question, please select "Accept" beneath that answer. This encourages user participation in this forum.

[#3]

@Matt Clarke, thanks for your answer, i am not aware about the file sequences.