Created 04-25-2023 09:10 AM
How to know when NiFi has done processing with all the flowfiles in the processor group? Also is there any way to notify external API?
Created 04-25-2023 01:57 PM
NiFi is a data in motion design. Processor components execute against the highest priority FlowFile in an inbound connection. An individual processor does not know how many FlowFiles to expect from any upstream component processors. So this becomes a challenge for downstream processors to know when all FlowFiles have been processed. Do you know or have a way to determine the number of FlowFile that will enter yoru process group?
Perhaps details of your use case may help a community member to suggest something for you. (how does data land in your process group, how much data, how often data enters the process group, etc...)
As far as your alert an external API, you could use the invokeHTTP processor to do that. The challenge here is knowing when to make that notification.
If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.
Thank you,
Matt
Created 04-25-2023 01:57 PM
NiFi is a data in motion design. Processor components execute against the highest priority FlowFile in an inbound connection. An individual processor does not know how many FlowFiles to expect from any upstream component processors. So this becomes a challenge for downstream processors to know when all FlowFiles have been processed. Do you know or have a way to determine the number of FlowFile that will enter yoru process group?
Perhaps details of your use case may help a community member to suggest something for you. (how does data land in your process group, how much data, how often data enters the process group, etc...)
As far as your alert an external API, you could use the invokeHTTP processor to do that. The challenge here is knowing when to make that notification.
If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.
Thank you,
Matt