Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi - to know whether a flow has suceeded or failed and also how to know at what part the failure has occured

Highlighted

NiFi - to know whether a flow has suceeded or failed and also how to know at what part the failure has occured

New Contributor

How to know whether a flow has succeeded or failed and also how to know at what part the failure has occurred in NiFi?

1 REPLY 1

Re: NiFi - to know whether a flow has suceeded or failed and also how to know at what part the failure has occured

@surajnag This is a great question.  Here are some of my ideas which I have used in development and production nifi flows.

 

  1. During development route all flow exceptions to an output port.  I call these EOL, or End of Line and increment them with #s, like EOL1, EOL2, EOL3.  I use these to hold failure, retry, original, no-retry, etc type outbound connections during testing.  As I move to production, some of these may be auto terminated, and some may remain.
  2. In production, I route ALL major points of failure which are not auto terminated to an output port called ERROR.  In my flows I sometimes have multiple ERROR output ports for difference purposes.   Depending on the use case, the error is sent to some type of event notification.  For Example:  send an email, or post to a slack channel.  In other cases these are routed to a process group that is in a stopped/disabled state.  Based on the error, I may make some change in the flow, and then enable/start the flow to create a method to REPLAY that flowfile.
  3. In reference to successful end of flow, I may just auto terminate, as I want to assume the flow is always on, and everything has succeeded if it is not routed as above in #1 or #2.  In other cases the bottom or end of the flow is also routed to some type of Event Notification.  However, be careful here, as you could create tons of notifications depending on your flow.


I hope some of these ideas are helpful.

 

 

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  

 

Thanks,


Steven

 


 


If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  


 


Thanks,



Steven

Don't have an account?
Coming from Hortonworks? Activate your account here