@RohanBajaj  This is a very old thread with a very old version of Apache NiFi from the very early days of NiFi's introduction of the load balancing capability through the connections.  I recommend you start a new community question with the specifics of any issue you are having to get the best possible assistance from the community members. Thank you, Matt ... View more

@donaldo71  I have not been able to identify a known issue that aligns with the description you have shared. That is an interesting sequence of events on a single FlowFile (SEND followed by clones).  Can you share the "Relationships" configuration of your putSQL processor? Make sure you have not checked the "retry" box on the "success" relationship.   Something you might want to try to see if same issue persists is to check the box for "retry"  on the retry relationship.   This allows the original FlowFile to remain in the inbound connection up to the configured number of retry attempts (default 10) before being routed to retry relationship.  I'd be curious of your observations post the above configuration change. Would you be willing to download the your flow definition json for this dataflow and share it? This is your full "SQL Statement" set in your ptSQL processor? UPDATE tbl SET status = 'proceed', startDate = GETDATE() WHERE messageId = ${messageId} Where are your utilizing those two attributes that go missing? can you share your UpdateAttribute processor configuration? Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@AlokKumar   You are correct that the ConsumePOP3 processor does not support an inbound connection to it.    Even if it did, the username and password fields do not support NiFi Expression language allowing you to pass either of those values in from the source FlowFile.   There aren't any other native processors that can support this dynamic credentials use case.   You would need to create a custom script that could be called by scripting processors or create your own custom processor.  ExecuteScript ExecuteProcess ExecuteGroovyScript The reason processors like ConsumePOP3 do not support inbound connections is because they are designed to execute continuously on a run schedule and produce an individual FlowFile for each new email message consumed.   So supporting an inbound connection raises the question... What do you do with the source FlowFile that you would use as the trigger?  Then you also have the challenge of continuously consumption.  You would need to keep producing an input FlowFile for each email account to make sure you keep consuming from each source account.   Plus this processor does not write any attributes to outbound FlowFile to distinguish which account message came from.    Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

Sorry for the delayed response. We were able to kind of resolve the issue by adding a retry on ListSMB and FetchSMB processors.    Number of Attempts : 2 Retry Back Off Policy: Penalize Retry maximum backoff period : 1 minute    To test the working we have scheduled it to run every 30 min. However, we are observing that whenever a retry happens the scheduler won't run  on scheduled time. Not sure how retry is affecting scheduler.   Thanks! ... View more

@Vishesh  Sorry, but there is not enough information to give a definitive cause for your issue.  There is typically full stack traces to go with these types of logged exceptions which can tell us more, but even then it may not be enough information still.   As i mentioned before there are numerous improvements between Apache NIFi 1.23 and the last Apache NIFi 1.x release version 1.28.  I'd strongly encourage you to upgrade to see if your issue persists.   You may be hitting this bug  NIFI-12232  which was addressed in versions 1.26+.  Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

Not so much resolved as worked around.  We had a large number of components on the canvas.  And we think we tracked the issue to timeouts in the cluster sync process.  (It was taking too long sometimes to get flow.xml sync'd between the nodes so it would stop.) The default timeout is 30 seconds.  We moved it up to 180 seconds. In Cloudera Manager, for the NiFi Configuration, search for properties which start with `nifi.cluster` to find the ones we changed. I also think things might have gotten better with more recent releases.  We have not seen this happen for quite some time. ... View more

@Frank168  I'd expect what you are seeing with the long sync times.  Sync interval has not impact in speed of the user an group syncs.  This simply controls how often NiFi is expected re-sync with ldap.  So in your case, NiFi would finding one sync and immediately start the next.  NiFi does a sync during startup as it loads the authorizers.xml, NiFi does not continue to load until that initial sync completes otherwise it can't do authorizations.   After that initial successful sync with ldap, NiFi will finish loading and only then will UI become available.   While running the sync interval will run in the background to re-sync from ldap in case any users or groups are added/removed/updated since last sync.    some users adjust this to sync every 24 hours instead of every 30 minutes.   This depends on how quickly you want NiFi to be aware of changes. As I commented before, you are loading 50,000+ users into NiFi heap memory (this impact the free heap available to the NiFi process for executing your dataflows on the canvas) and then redoing that every sync interval.  This was not how NiFi ldap-user-group-provider was intended to be used.   The only users and groups that should be loaded in to NiFi are those you plan to establish authorization policies for and will be accessing your secured NiFi. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Krish98  Unfortunately, I have no "Best Practice" recommendation for the use case you have shared.  It is not a use case I have ever setup before. Also want to share that in Apache NiFi 2.4+ version a new JWTBearerOAuth2AccessTokenProvider controller service was introduced.  While not a solution to you query, I wanted to share this with you.  Apache NiFi jira: NIFI-14380 NOTE: The Apache NiFi 1.x major release line is End-Of-Life now.  There will be no future releases of the 1. major release line.  There is no direct upgrade path from Apache NiFi 1.x to Apache NiFi 2.x.  You'll need to migrate your dataflows from 1.x to 2.x. For our Cloudera Flow Management licensed users, we provide tooling to assist with migrating dataflows from Flow Management versions based on Apache NiFi 1.x to Flow Management versions based on Apache NiFi 2.x.   Cloudera Flow Management 2.x also includes many of the components deprecated and no longer included in the Apache NiFi 2.x release line.  Thank you, Matt ... View more

Thank you @mat . ... View more

@hckorkmaz01  @vafs provided a couple excellent options already. As a third option, you could utilize the SiteToSiteStatusReportingTask.  You can configure this NiFi reporting task to send metrics for only connections and you can add a regex to limited the number of connections returned (this requires renaming the desired connections so you have more granular control with the regex).  This Reporting task can be configured to send to a remote input port on on the same NiFi.   This Reporting task will execute the query on each host in your NiFi Cluster and send results to the the desired Site-To-Site remote input port.   So in a three node NiFi cluster three FlowFiles will be produced and sent with Json content for connection(s) you filtered on.   This way you can see what is queued per node.  Limitations to monitoring connections in this way exist.  Keep in mind that you are grabbing a snapshot at one specific moment in time.  So if the downstream component of this monitored connection is running, the connection stat would have likely changed by the time you got the last status details.   Now if you are using this to monitor flow dead-ends where you don't expect FlowFiles to ever accumulate, that is perhaps a valid use case. Example json output placed in FlowFile: [ { "statusId" : "27b01368-8280-4e5e-ac22-0e749c46fe17", "timestampMillis" : 1771336855410, "timestamp" : "2026-02-17T14:00:55.410Z", "actorHostname" : "nifi-node-1", "componentType" : "Connection", "componentName" : "Monitored-success", "parentId" : "603017c1-0197-1000-0000-000013171e7c", "parentName" : "test", "parentPath" : "NiFi Flow / test", "platform" : "nifi", "application" : "NiFi Flow", "componentId" : "508a2293-019a-1000-ffff-fffff15dc582", "sourceId" : "5089d4ab-019a-1000-ffff-ffff9b682ab3", "sourceName" : "UpdateAttribute", "destinationId" : "4dc23b89-25b0-1eff-b974-6fe5425c283f", "destinationName" : "UpdateAttribute", "maxQueuedBytes" : 0, "maxQueuedCount" : 0, "queuedBytes" : 30720, "queuedCount" : 30, "inputBytes" : 0, "inputCount" : 0, "outputBytes" : 0, "outputCount" : 0, "backPressureBytesThreshold" : 1073741824, "backPressureObjectThreshold" : 10000, "backPressureDataSizeThreshold" : "1 GB", "isBackPressureEnabled" : "false" } ]   IMPORTANT NOTE: I'd also like to point out that Apache NiFi 1.18 was released back in 2021 and newer releases have addressed many bugs and CVEs.  I strongly encourage you to transition to a newer version of Apache NiFi.  If you aren't ready to migrate to the newer Apache NiFi 2.x major release versions, you can still easily upgrade to last Apache NiFi 1.x major release version (1.28.0). Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more