@fy-test  The NiFi node that disconnects due to a flow mismatch should inherit the cluster flow when it attempts to rejoin the cluster.  The only time this is not possible is if the cluster flow includes change that would result in dataloss. Example: Cluster flow has a connection removed that on the connecting node still has queued FlowFiles. NiFi has not feature to force removal/archive of a flow.json.gz on a disconnected node. You could file an Apache NiFi improvement jira here: https://issues.apache.org/jira/projects/NIFI But first step is to identify why your node is not able to inherit cluster flow and rejoin the cluster.  What is the exception logged when it attempts to rejoin the cluster? Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

hello @Darryl  the ldap exception shared indicates your ldap does not support anonymous access and expects authentication:   Caused by: javax.naming.NoPermissionException: [LDAP: error code 50 - No user currently bound] 2025-02-05 09:20:33 at java.naming/com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3268)   using the "SIMPLE" strategy would use the manager DN and Manager password to connect to the ldap.  "LDAPS" strategy would still require manager DN and Manager password, but would also require TLS properties configured (truststore only if one one may TLS is required by your ldap or both keystore and truststore if mutual TLS exchange is required by your ldap). LDAPS strategy requires using the ldaps://<url>:<ldaps port>.  "START_TLS" is rarely used, but has same requirements as LDAPS strategy; however, connection is initially made insecure and later redirected to secure connection url. https://nifi.apache.org/nifi-docs/administration-guide.html#ldapusergroupprovider Also: Your user search base configuration is till not valid, but that is not related to your ldap connection exception. Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@jfs912  You should not be configuring your NiFi with larger then necessary heap.  Doing so just leads to very long stop-the-world garbage collection events.   The simple flow you have described would use very little heap memory. So you have a 5 node cluster and each nodes has files in some local directory that each node is pulling from? Is that local directory a mounted directory that is mounted to all nodes or each nodes has its own set of files in the local directly from which getFile is pulling from? Are you seeing backpressure being applied on any of the connections between your processors?  When backpressure is being applied to the upstream processor, NiFi will not schedule that upstream processor until that backpressure is removed. If you can tolerate some latency in your dataflow, you can get better throughput performance with some processors by increasing the Run Duration as well. Dataflow design best practices and designs can also improve performance and better load distribution across all the nodes in your cluster.  You want to minimize as much as possible one node doing bulk of the work load. Adjusting concurrent tasks has multiple elements to it. What is the current CPU load average on each of yoru 5 servers?  First need to know if there is capacity to run more parallel threads. How large is the configured timer driven thread pool in NiFi?  It is from this configured thread pool that all concurrent tasks used by processor components comes from.  If this pool is small, adding more concurrent tasks to processors will improve nothing.  Ability to increase the size of this thread pool is dependent on node's cpu load average.  Thread pool is also applied per node.  So when set to 10 that is 10 threads per each node in your 5 node cluster.    If cpu load average is not high and you increase the size of the Timer Driven Thread pool, you'll want to make small incremental changes to the concurrent tasks on processor and monitor impact on CPU load average. Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more