what does 'Penalty duration' mean in nifi processor settings

zhangweigang20 — Fri, 02 Sep 2016 16:06:58 GMT

in nifi processor settings, there is a property

Penalty duration:The amount of time used when this processor penalizes a FlowFile.

what does 'processor penalizes a FlowFile.' mean?

Re: what does 'Penalty duration' mean in nifi processor settings

JoeWitt — Fri, 02 Sep 2016 17:54:18 GMT

If the processor in question believes there is something about a given flowfile that is temporary and may resolve itself it will mark the flowfile at penalized. When it routes that penalized flowfile to some outgoing connection those penalized flowfiles will not be accessible to the processor that might consume it until the penalty period expires. You can read a bit more about that here https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#settings-tab

A good example where this is useful is to consider delivery of flow files to some remote system using PutSFTP. It is common to route 'failures' of PutSFTP back to itself so it will keep trying. But, sometimes there can be conflicts like filenames on the remote server that match so you want to wait until they clear out and try again. In this case penalization lets us operate on other data while we put the problematic flowfiles off to the side. It's all just part of helping ensure the most productive action possible can happen and we're not just sitting there pounding the remote system with the same flow file over and over.

Re: what does 'Penalty duration' mean in nifi processor settings

zhangweigang20 — Mon, 05 Sep 2016 07:30:11 GMT

Thanks for your reply. I get it.

question Re: what does 'Penalty duration' mean in nifi processor settings in Support Questions

what does 'Penalty duration' mean in nifi processor settings

Re: what does 'Penalty duration' mean in nifi processor settings

Re: what does 'Penalty duration' mean in nifi processor settings