Nifi Workflow to store count of success and failure for multiple route

rahul_ars — Sun, 16 May 2021 14:22:58 GMT

Hi,

We have scenario to read file feed containing different data schema for lines and then write to multiple tables. We are trying to setup error handling and consolidated all success and failure count for different types of data. our data flow is almost as follows,

getFile -> splitText -> RouteText -> ConvertToJson -> JsonToSql -> PutSql

Can you please help with some suggestion. Thanks in advance.

Regards,

Rahul

Re: Nifi Workflow to store count of success and failure for multiple route

MattWho — Mon, 17 May 2021 12:34:12 GMT

@rahul_ars

You can use the "UpdateCounter" [1] processor to create counters and update the count on them (up and down). The "Counter Name" property accepts NiFi EL, so you can use attribute son FlowFiles to dynamically update a counter uniquely by FlowFile traversing same dataflow path. This will allow you to consolidate within your dataflow(s) to reduce the number of processors needed. Fewer processors can lead to better performance through reduced resource utilization.

The "Counters" UI is found under the NiFi Global menu in the upper right corner of the NiFi UI.

[1] http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.UpdateCounter/index.html

If you found this help with your query, please take a moment to login and click accept on this solution.
Thanks,

Matt

question Re: Nifi Workflow to store count of success and failure for multiple route in Support Questions

Nifi Workflow to store count of success and failure for multiple route

Re: Nifi Workflow to store count of success and failure for multiple route