Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Notify external api/agent after Nifi is done processing all the flowfiles in the processor group

avatar
Explorer

How to know when NiFi has done processing with all the flowfiles in the processor group? Also is there any way to notify external API?

1 ACCEPTED SOLUTION

avatar
Super Mentor

@kishan1 

 

NiFi is a data in motion design. Processor components execute against the highest priority FlowFile in an inbound connection.  An individual processor does not know how many FlowFiles to expect from any upstream component processors.  So this becomes a challenge for downstream processors to know when all FlowFiles have been processed.   Do you know or have a way to determine the number of FlowFile that will enter yoru process group?

Perhaps details of your use case may help a community member to suggest something for you. (how does data land in your process group, how much data, how often data enters the process group, etc...)

As far as your alert an external API, you could use the invokeHTTP processor to do that.  The challenge here is knowing when to make that notification.

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt

View solution in original post

1 REPLY 1

avatar
Super Mentor

@kishan1 

 

NiFi is a data in motion design. Processor components execute against the highest priority FlowFile in an inbound connection.  An individual processor does not know how many FlowFiles to expect from any upstream component processors.  So this becomes a challenge for downstream processors to know when all FlowFiles have been processed.   Do you know or have a way to determine the number of FlowFile that will enter yoru process group?

Perhaps details of your use case may help a community member to suggest something for you. (how does data land in your process group, how much data, how often data enters the process group, etc...)

As far as your alert an external API, you could use the invokeHTTP processor to do that.  The challenge here is knowing when to make that notification.

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt