Support Questions
Find answers, ask questions, and share your expertise

FlowFile's timestamp upon processor output

Solved Go to solution

FlowFile's timestamp upon processor output

Contributor

In NiFi, is there an existing FlowFile attribute that can tell when it exited (or when it was outputted by) a certain processor? I am trying to measure end-to-end processing time of a workflow.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: FlowFile's timestamp upon processor output

This type of information is typically stored in provenance data... You can use the SiteToSiteProvenanceReportingTask to get access to provenance events in JSON format and then filter the events to find the ones you are interested in. Each provenance event should have an event time which is the time the event was reports, as well as the lineage start time which is the time of the first event in the given lineage. So event time - lineage start time would be the time it took to get to current event.

View solution in original post

1 REPLY 1

Re: FlowFile's timestamp upon processor output

This type of information is typically stored in provenance data... You can use the SiteToSiteProvenanceReportingTask to get access to provenance events in JSON format and then filter the events to find the ones you are interested in. Each provenance event should have an event time which is the time the event was reports, as well as the lineage start time which is the time of the first event in the given lineage. So event time - lineage start time would be the time it took to get to current event.

View solution in original post