@ajignacio  The PutMarkLogic processor is not a component bundled and shipped with Apache NiFi, so I am not familiar with it.  You may want to raise your issue directly with those who developed this connector and include your Apache NiFi version specifics as well: https://github.com/marklogic/nifi/issues Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@jirungaray  The DistributedMapCacheServer controller service sets up a cache server which will keep all cached objects in NiFi's JVM heap memory.  This cache is lost if the controller service is disabled/re-enabled or if NiFi were to restart unless the "Persistence Directory" is configured.  The persistence directory is some local disk directory where cache entries are persisted in addition to those cache entries also being in Heap memory.  The persistence to disk allows the in memory cache to be reloaded if the cache server is disabled/re-enabled or NiFi is restarted.  I assume this is the cache server you are currently using. Matt ... View more

@Bern  Unfortunately there is not enough information here to understand exactly what is going on. The only exception shared was related to an attempt to terminate a thread on some processor.  As far as why you see this, there is not enough information to say.   It could be a bug in an older version, could be load issue, could be thread pool exhaustion, etc. Observations and questions: You are running with a very old version of Apache NiFi release 6+ years ago and one of the first releases to offer the Load-Balanced connections feature which was very buggy when it first was introduced.  You would greatly benefit from upgrading for security fix and bug fixes reason. You see to be using the load-balanced connections excessively.  It makes sense to redistribute NiFi FlowFiles in connections after your executeSQL processors, but i see no value in redistributing after RouteOnAttribute or on the failure connections.  This just adds excessive and unnecessary network traffic load. I see you have ~1400 running components and a queue of ~265,000 FlowFiles. What is the CPU load average on each of yoru nodes and how many nodes do you have in your NiFi cluster? What java version is being used?  Are Garbage Collection (GC) stats healthy.  How often is GC (partial and full) running?  How long is spent on GC?  Any other ERROR in your nifi-app.log? Have you taken any thread dumps when you are having issues with processor component threads?  What did you observe there? Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

Thank you for the detailed answer. ... View more

Thank you @MattWho for details. As you mentioned, I will post my usecase in the jira.  Thanks for your help! ... View more

@ajay32 Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks. ... View more

@AlokVenugopal  Welcome to the community. What you are encountering is an authorization issue and not an authentication issue.  NiFi is accepting your token issued through your application login, but then authorization does not exist for the user identity derived from you token. In NiFi, after successful authentication, the user identity is passed to the NiFi authorizer to determine what NiFi policies have been authorized for that user identity.  When using yoru application's token, this result in no authorization found because neither the user Identity or any known groups that user identity belongs to are authorized for the required policy. identity[kLM-4Eld2dZnX_dD3iB0df2fTvXQxa1J2ffdLoK-ozas], groups[]  Supporting the user "unique id" would require that NiFi's authorizer contained that unique id and it was authorized to the necessary NiFi policies.  Authorizing users based in these unique id does not make much sense in NiFi as it would be error prone and difficult to manage authorization.  An Admin would need to know what user these unique ID map to in order to setup authorization successfully. The first option would be modifying your app so that the returned token contain and ID that matches the user identity similar to what NiFi does. Assuming this "unique id" does not change and is always the same for the specific user, perhaps you can work around this creatively within NiFi through group based authorization. This would requiring using the file-user-group-provider within the NiFi authorizers.xml. This will allow you to manual add user identities and group identities.   So you create a new group such as "username" via the NiFi UI.  You then add your existing user (the one that successfully gets authorized when you authenticate through NiFi) to this new group.  You then add a new user identity for that "unique id" and make that new user a member of that same group via the NiFi UI.   Now authorize the group to whichever policies are necessary.    Now no matter if your user authenticates via NiFi to get token or through your app to get a token, the user will successfully be authorized via the shared group membership. Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

Thank you @MattWho .  Thanks for brief explanation. I understand the loop is causing large number of queue. Let me redesign the flow.  Thanks !  ... View more

@Jaydeep Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks. ... View more