@Shrink  Welcome to the community. Unfortunately,  your image attachment is not present for review and there is not enough detail to provide any detailed suggestions here. NiFi processors like ScriptedFilterRecord and QueryRecord execute against the content of the NiFi FlowFile from the upstream connection.  The ScriptedFilterRecord processor has three possible downstream relationships (Success, Failure, and Original).  I guess my first question would be how are you routing these three relationships? Assuming your downstream connection does not contain both the original and success relationships in it, have you inspected the content on the connection FlowFiles prior to starting the QueryRecord processor to make sure it is what you expect? Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@dan_lucas  From the exception, this appears to be a configuration issue most likely. You'll want to verify the NiFi Expression Language statement used in the putFTP processor's "Remote Path" property. I assume you have something configured there like ${absolute.path}/${airlinename} in that property? If you manually connect to the FTP server can you successfully navigate the path? Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@ajay32  Welcome to the Community! Unfortunately, I don't think you have provided enough detail to answer your questions. NiFi is typically is used to build dataflows that are always running, so a clear well defined use case for you dataflow may help the community to provide you with responses. "Display all the configured integrations available to users" <-- not clear on yoru ask here.  What do you mean by "configured integrations"?  Is an integration an End-to-End NiFi Dataflow build on the NiFi canvas? Show execution details: Start Time - Assuming NiFi dataflow, how are your processor components being scheduled?  (Cron driven?, Timer Driven?) End Time - Downstream processors are not typically scheduled using cron, but rather always running using timer driven to ensure they can process NiFi Flow Files as they are received.  There is no communication between one processor and another.  NiFi processors simply execute against a NiFi FlowFile and that FlowFile is passed to the next downstream processor.  That downstream processor is going to be unaware of how many FlowFiles to expect from the upstream processor.  Duration - The execution of your first Processor in a dataflow will produce a NiFi FlowFile with a timestamp.  You could compare that timestamp with current time at end of your complete dataflow execution to calculate how long it took for that FlowFile to process through all NiFi processor components. Status - What constitutes a "success" and "failure" in your dataflow / use case? Sub-items - What is a "sub-item" in your dataflow? Track all integrations: You can build each unique dataflow in a NiFi process group and enable per process group logging by configuring a unique "Log File Suffix" within the process group configuration. NiFi provenance is also an option.  If you are dealing with multiple unique dataflows, modifying processor names so that those in one complete dataflow share the prefix ("flow1_<processor name>") would make it easier to correlate all events related to a single datflow.  There are a few Provenance NiFi reporting tasks that can be used. Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Emery  No argument with you there.  A rest-api call fetches a json of all counters which you then need to parse to extract the specific counter value you are looking for.   The NiFi counters where never intended to be used for anything more then simple counts during flow development testing. Matt ... View more

@Emery  1. Counters: NiFi counters only exist in NiFi heap, so values are lost if NiFi is restarted.  2. Parameter Contexts are static defined values intended use across many components thus along a single place to update when needed. I don't know what your use case is for these counters, but perhaps you could use the following processors to maintain a count and be able to fetch the value when needed in your dataflows: PutDistributedMapCache 1.28.0 FetchDistributedMapCache 1.28.0 So you can Fetch current value from cache, update it, and put replace value in cache.  Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt   ... View more

@AlokVenugopal  Welcome to the community. What is being logged in the nifi-user.log when you attempt to use this token issued by through authentication through a different service?  The log entries should show the exact user identity string. I see from your shared nifi.properties that you are using OpenID Connect to authenticate your users with your NiFi. This provider is redirecting through NiFi to your Microsoft SSO provider login.    From your authorizers.xml we see you are using the AzureGraphUserGroupProvider user-group-provider. Can you try changing:   <property name="Claim for Username">email</property>   to:   <property name="Claim for Username">upn</property>   So that this aligns with this configuration in your OpenID COnnect: nifi.security.user.oidc.claim.identifying.user=upn   Take note of what is observed currently and after making this change in the nifi-app.log for your authenticated user when you authenticate via the NiFi service. Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@fy-test  Welcome to the community. What are the reasons being given in the NiFi elected cluster coordinator nifi-app.log for the disconnection or from the disconnected node's nifi-app.log? No matter which NiFi node you are connected to any, change request must be sent the elected "Cluster Coordinator" that replicates that request to all connected nodes.   If any of the nodes that has been requested to make the change fails to respond in the node will get disconnected.  The elected "Primary node" is the node on which any primary only scheduled processor components will run. It is also important to understand that which node is elected as the "Primary" or "Coordinator" can change at any time.   I don't think forcing all your users on to the Primary node is going to solve your issue.  Even with a node disconnection caused by a failure of the request replication, the disconnected node should attempt to reconnect to the node and inherit the cluster flow if is it different from the local flow on the connecting node. You should also be looking at things like CPU load average, Heap usage, and Garbage collection stats on your primary node versus the other nodes. Perhaps adjust max timer driven thread pool sizes or adjusting timeouts would be helpful. Cluster Node Properties   How well are your dataflow designs distributing the load across all nodes in your cluster? Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more