Nifi workflow success/failure record count

[#1]

Hi All,

I have a scenario where i need to consume data from Elastic search and do some processing (like filter, enriching etc). I have to store this processed information to Redis. I have implementd the flow like Scroll ES Processer -- > Execute script . This is working well.As a part of ETL operation, i have to record total pass & failed record count at end of the process

few question,

1. how can i track total count of success and failure

2. how can i implement the workflow to wait until all the data consumed from elastic search then update the success and failure count to DB?

your help will be much appreciated.

Thanks,

SK

[#2]

Hi @senthilkumar kalimuthu

For 1 look at UpdateCounter processor https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache...

For 2 look to wait notify processors. They are similar to P and V functions if you are used to Semaphores

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache...

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.4.0/org.apache...