@manishg  The Record Reader and record Writer controller services are not responsible for tracking counts.  This is handled within your custom processors code. I am not sure what your custom processor does and whether it makes sense to track "record processed" or some other record based stat, but you can look at the github code for other processors like PartitionRecord to see how RecordCount is handled. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@davehkd  Of your NiFi cluster is secured you'll need to make sure that the load balancer is configured with sticky sessions (also known as session persistence).  This is needed because NiFi authentication (except certificate based mutual TLS authentication) issues a client and server side token.  The issued client token gets passed by the client (browser) with every subsequent request made to NiFi.  The corresponding server side token only exists on the specific NiFi node that handled the authentication.  So if your LB routes subsequent requests to a different node, authentication will fail for that request. Many users setup LBs in front of NiFi so there is one URL that can direct to any number of nodes in the NiFi cluster that are all capable fo handling authentication and authorization.  This ensures ease of access for example when a node in the cluster is down. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt   ... View more

@Soli  There is likelihood that once you upgrade that you will have ghosted processors (happens when you are using a processor that no longer exists in the newer release) or invalid components (happens when new version of same component class introduces new required properties that need to be configured). Something you may want to try is standing up a different NiFi on 1.23  with autoresume.state set to false in the nifi.properties. drop your flow.xml.gz in to that NiFi and start it.  Everything will load up in stopped state.  At least then you can validate al your processors are valid and none are ghosted.  If this standalone can reach yoru redis, make sure you don't start processors that use it as it can mess up state recorded in your prod NiFi.   You also did not mention if you have a standalone NiFi or a NiFi cluster setup.  In a NiFi cluster, Zookeeper is also used by some components to store cluster wide state.  So would not utilize same ZK in your test cluster setup used to check for component issues. At least you will be able to rule out flow design issues and be able to make note of what needs to be fixed in your production upgrade prior to doing the upgrade. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt   ... View more