Support Questions

Find answers, ask questions, and share your expertise

How to keep track of flowfiles going through a failure relation through nifi data provenance.

avatar
New Contributor

Hi, 
I have a requirement where I want to check any flow files going through failure relationships through data provenance directly.

2 REPLIES 2

avatar
Community Manager

@ramMuka Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi experts @cotopaul @SAMSAL and @MattWho  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar

Assuming that there would be a way to check for flow files which went through failure, how would you extract them? Manual search within the Data Provenance Menu from within the GUI? Or using the REST API?
Now, what I do know is that you cannot identify those files that easy, because they do not write any specific lines within the Data Provenance. If you have a processor right after the failure queue (or if you terminate the failure queue in the processing processor) you can query using that Component ID and identify the type = DROP, meaning that those files have been "processed" (Indicates a provenance event for the conclusion of an object's life for some reason other than object expiration).
More about the types can be found here: https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.1.2/bk_user-guide/content/provenance_events.html