Support Questions

Find answers, ask questions, and share your expertise

FlowFile's timestamp upon processor output

avatar
Expert Contributor

In NiFi, is there an existing FlowFile attribute that can tell when it exited (or when it was outputted by) a certain processor? I am trying to measure end-to-end processing time of a workflow.

1 ACCEPTED SOLUTION

avatar
Master Guru

This type of information is typically stored in provenance data... You can use the SiteToSiteProvenanceReportingTask to get access to provenance events in JSON format and then filter the events to find the ones you are interested in. Each provenance event should have an event time which is the time the event was reports, as well as the lineage start time which is the time of the first event in the given lineage. So event time - lineage start time would be the time it took to get to current event.

View solution in original post

1 REPLY 1

avatar
Master Guru

This type of information is typically stored in provenance data... You can use the SiteToSiteProvenanceReportingTask to get access to provenance events in JSON format and then filter the events to find the ones you are interested in. Each provenance event should have an event time which is the time the event was reports, as well as the lineage start time which is the time of the first event in the given lineage. So event time - lineage start time would be the time it took to get to current event.