@Mustafa Ali Qizilbash Do you experience same issue with a different browser? What version of Firefox and NiFi are your using? - Have tried using the developer tools (Network) available in the Firefox browser to check for any Java Script errors? From Firefox menu in upper right corner select "web developer" and then "network". A new window will open. Then try reloading NiFi page. You should be able to see where it stalls on loading the page. - Thanks, Matt ... View more

@David Miller The use of the distributedMapCache controller service by ListSFTP processor is used to store state information and has nothing to do with distribution of FlowFiles to all nodes in a NiFi cluster. - The ListSFTP processor should still only be scheduled to run on primary node only. Then fed to a RPG to redistribute the listed Flowfiles to all nodes before a FetchSFTP processor retrieves the content. - Zookeeper is used to elect a primary node in your NIFi cluster and from time to time a new primary node may be elected. The purpose of the distributeMapCache is so the last known state of the running listSFTP processor can be stored somewhere all nodes can access it. That way when a primary node change occurs, the ListSFTP processor that starts running on a different node does not list files the previous primary node ListSFTp processor already listed. - Thanks, Matt ... View more

@Hans Feldmann - You are hitting a known bug in MiNiFi 0.4.0: https://issues.apache.org/jira/browse/MINIFI-435 - Fix is part of MiNiFi 0.5.0 or you can roll back to MiNiFi 0.3.0 which i understand works as well. - Thank you, Matt ... View more

@Davide Vergari @srinivas p - Let me explain what is going on here so you can understand why the configuration change made helps in some cases and not others. - When you add a component to the canvas of a NiFi cluster the following steps are performed. 1. That request is forwarded to the current elected cluster coordinator on behalf of the user who added the component. 2. The cluster coordinator then replicates that request to all the nodes connected in the cluster. (the nifi.cluster.node.protocol.threads=10 setting dictates how many concurrent request can be made, so larger clusters will need to have this value increased) 3. Each node must then make the change and respond back to the cluster coordinator. - While this process is consistent for every such replicated request, not all changes are equal. The action of selecting a bunch of components on the canvas copying and pasting them or instantiating a large template to the canvas also constitutes a single replication requests instead of many requests. These component bundles are referred to as snippets in NiFi. This action is not asynchronous. this means that each node must add every component (processors, connections, controller services, etc..) from this snippet before it responds to the cluster coordinator. Depending on size of the snippet and load on the server, this request may very likely exceed to configured timeout. This results in those nodes that timed out being disconnected from cluster. - Increasing the timeouts allows more time for these snippets to be instantiated and response to be received. Because there is no way to know how large these snippets are, the timeout setting that works for one user may not work for others. - Two things to keep in mind here: 1. How many node requests can be made concurrently (as shown above default is 10). Using a 16 node NiFi cluster as an example, 10 nodes must respond before the other 6 even get the request. so increase this value. 2. The nifi.cluster.node.connection.timeout and nifi.cluster.node.read.timeout can be set to much higher values. Even settings these timeouts to 2 - 5 minutes does not mean every request will take that long. It simply means that you will allow that much time before the cluster coordinator makes the decision to disconnect the node due to timeout. - There is work on the roadmap to redesign these types of replication requests in to asynchronous type requests eventually. Once that happens user will not need such high timeouts configured. - Thank you, Matt ... View more