Support Questions
Find answers, ask questions, and share your expertise

Monitoring processing time for flowfiles in NiFi

New Contributor


I am looking for a way to monitor flow files as they are ingested into the start of the flow and end up being published to Kafka. I have looked at the following link (, but It's not exactly what I am looking for.

I would basically like to monitor the flow files as they go trough each processor and if they take longer than a certain amount of time to reach a certain point, send out a mail notification.

I was thinking of adding a RouteOnAttribute in multiple locations in the flow, but I am not sure if this is the best way to do this. I am also not sure if the time the flowfile came in, will be kept as an attribute throughout the whole flow or if a new time is generate each time the flow file hits a new processor.


Super Guru

You can use the SiteToSiteProvenanceReportingTask for this. Filter the reporting task to only emit events at the "certain point" you mention above. Each event has a "timestampMillis" and "lineageStart" field, you should be able to route on the difference of the two using QueryRecord, with something like:

SELECT * FROM FLOWFILE WHERE timestampMillis - lineageStart > 60000

Which should emit a flowfile containing all events for which the associated entity (in this case, the flow file in the system) has been in the flow for over a minute.


@mburgess is there a way i can access the lineage duration from a update attribute processor. I like to store this in a database along with other meta information for analysis. 


@Althotta As this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post. Thanks!


Diana Torres,
Community Moderator

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:
; ;