@David Miller I don't understand the concern with a "disconnected node" still receiving data from a load-balancer? Just because a node is disconnected does not mean it is not still running its dataflow(s). If the NiFi service or hosting server is down, their will be no running listener to receive data from the load-balancer, so the LB should failover to a different server. ... View more

@Rakesh S - Short answer: The Cluster Coordinator has no role in data distribution within a cluster. Each NiFi node only works on data it specifically receives. - ... View more

@umang s Based on your flow design above, it looks like you are trying to route FlowFiles by comparing attribute between two different FlowFiles? That will not work. NiFi is looking for both ${temp_array} and ${category} to exist on same flowfile being evaluated by the RouteOnAttribute processor. ... View more

@Derek Calderon - Short answer is no. The ExecuteSQL processor is written to write the output to the FlowFile's content. - There is an alternative solution. You have some processor currently feeding FlowFiles to your ExecuteSQL processor via a connection. My suggestion would be to feed that same connection to two different paths. The first connection feeds to a "MergeContent" processor via a funnel and the second feeds to your "ExecuteSQL" processor. The ExecuteSQL processor performs the query and retrieves the data you are looking for writing it to the content of the FlowFile. You then use a processor like "ExtractText" to extract that FlowFIles new content to FlowFile Attributes. Finally you use a processor like "ModifyBytes" to remove all content of this FlowFile. Finally you feed this processor to the same funnel as the other path. The MergeContent processor could then merge these two flowfiles using the "Correlation Attribute Name" property (assuming "filename" is unique, that could be used), min/max entries set to 2, and "Attribute Strategy" set to "Keep All Unique Attributes". The result should be what you are looking for. - Flow would look something like following: Having multiple identical connections does not trigger NiFi to write the 200 mb of content twice to the the content repository. a new FlowFile is created but it points to the sam content claim. New content is only generated when the executeSQL is run against one of the FlowFiles. So this flow does not produce any additional write load on the content repo other then when the executeSQL writes its output which i am assuming is relatively small? - Thank you, Matt ... View more

@Henrik Olsen - NiFi's various timeout settings are very aggressive. They are more ideal for a standalone NiFi instance running a fairly simple dataflow. In a NiFi cluster the following timeouts should be increased: nifi.cluster.node.connection.timeout=5 secs (increase to 30 secs) nifi.cluster.node.read.timeout=5 secs (Increase to 30 secs) nifi.zookeeper.connect.timeout=3 secs (increase to 60 secs) nifi.zookeeper.session.timeout=3 secs (Increase to 60 secs) - A restart of NiFi will be needed after making these changes. Another thing you could do when this condition is present is to use the browser developer tools to try to catch what action is timing out. Are you seeing a lot of full garbage collection occurring (if these stop-the-world events are long enough, it can also cause this). - Thank you, Matt ... View more

@Derek Calderon - Sorry to hear that. I did share this HCC link with a few devs I know if they have time to assist. - Thanks, Matt ... View more