question Thoughts on NiFi error handling in Support Questions

question Thoughts on NiFi error handling in Support Questions https://community.cloudera.com/t5/Support-Questions/Thoughts-on-NiFi-error-handling/m-p/236702#M198515 Hey guys!Hope you are having a good day! I have a couple of questions regarding error handling in nifi. I found following links to be very useful to learn more about nifi error handling :<UL> <LI><A href="https://community.hortonworks.com/questions/77336/nifi-best-practices-for-error-handling.html" target="_blank" rel="nofollow noopener noreferrer">https://community.hortonworks.com/questions/77336/nifi-best-practices-for-error-handling.html</A></LI><LI><A href="https://community.hortonworks.com/articles/76598/nifi-error-handling-design-pattern-1.html" target="_blank" rel="nofollow noopener noreferrer">https://community.hortonworks.com/articles/76598/nifi-error-handling-design-pattern-1.html</A></LI><LI><A href="https://pierrevillard.com/2017/05/11/monitoring-nifi-introduction/" target="_blank" rel="nofollow noopener noreferrer">https://pierrevillard.com/2017/05/11/monitoring-nifi-introduction/</A></LI></UL>So regarding error handling there seem to be two general ways to go:<UL> <LI>flows that you want to retry</LI><LI>flows that you don't want to retry</LI></UL>Flows that you want to retry Count the number of retries and if a certain number of retries is reached send an email to the admin. After that wait by using a disabled processor:<A href="https://community.cloudera.com/legacyfs/online/attachments/93706-screenshot-from-2018-11-22-14-12-33.png" target="_blank"><img src="https://community.cloudera.com/t5/image/serverpage/image-id/14348i5A9457EC2241BB58/image-size/medium?v=v2&px=400" role="button" title="93706-screenshot-from-2018-11-22-14-12-33.png" alt="93706-screenshot-from-2018-11-22-14-12-33.png" /> </A>Flows that you don't want to retry No need to count here, simply send an email and wait for the admin to fix the problem.<A href="https://community.cloudera.com/legacyfs/online/attachments/93707-screenshot-from-2018-11-22-14-14-05.png" target="_blank"><img src="https://community.cloudera.com/t5/image/serverpage/image-id/14349i1F508151B7A204A1/image-size/medium?v=v2&px=400" role="button" title="93707-screenshot-from-2018-11-22-14-14-05.png" alt="93707-screenshot-from-2018-11-22-14-14-05.png" /> </A>In both cases, we do not want to drop data that is routed to e.g. failure. Instead we route the data to a disabled processor, to queue it. After fixing the problem, the admin can enable the processor and the data can be processed again. Here is full example (a flow to retrieve an authentication token)<A href="https://community.cloudera.com/legacyfs/online/attachments/93708-screenshot-from-2018-11-22-14-19-15.png" target="_blank"><img src="https://community.cloudera.com/t5/image/serverpage/image-id/14350i571817CD60618398/image-size/medium?v=v2&px=400" role="button" title="93708-screenshot-from-2018-11-22-14-19-15.png" alt="93708-screenshot-from-2018-11-22-14-19-15.png" /> </A>What are your thoughts on that? Do you think this kind of error handling is correct/ or can be improved? What bothers me is, that the ERROR process group will be all over the place. However I do not see any way to centralize it (since we have to deal with each failure flow individually, in order not to loose data).<A href="https://community.cloudera.com/legacyfs/online/attachments/93708-screenshot-from-2018-11-22-14-19-15.png" target="_blank"></A><img src="https://community.cloudera.com/t5/image/serverpage/image-id/14351i5F161D5695B8DCC8/image-size/medium?v=v2&px=400" role="button" title="93708-screenshot-from-2018-11-22-14-19-15.png" alt="93708-screenshot-from-2018-11-22-14-19-15.png" /><img src="https://community.cloudera.com/t5/image/serverpage/image-id/14352iDF9E9433E4E5D798/image-size/medium?v=v2&px=400" role="button" title="93707-screenshot-from-2018-11-22-14-14-05.png" alt="93707-screenshot-from-2018-11-22-14-14-05.png" /><A href="https://community.cloudera.com/legacyfs/online/attachments/93706-screenshot-from-2018-11-22-14-12-33.png" target="_blank"></A><img src="https://community.cloudera.com/t5/image/serverpage/image-id/14353iD213E94C9DB7B752/image-size/medium?v=v2&px=400" role="button" title="93706-screenshot-from-2018-11-22-14-12-33.png" alt="93706-screenshot-from-2018-11-22-14-12-33.png" /> Sat, 17 Aug 2019 23:28:48 GMT artworkad 2019-08-17T23:28:48Z