@Mauro Beltrame-
Keep in mind that every node in your NiFi cluster runs its own copy of the flow.xml, has its own set of repositories, and works on its own set of FlowFiles. When a "primary node" change occurs, this does not mean that FlowFiles being processed on old primary node are moved over to the new primary node. It is still the responsibility of the old primary node to finish processing FlowFiles on that node. The "primary node" execution configuration setting on processors simply sets whether this processor should be scheduled to execute on all nodes at same time or just get scheduled on which ever nodes is currently elected the primary node. This is important for non-cluster friendly protocols that some processors use (i.e listSFTP, ListFIle, etc...).
-
Keep in mind that only processors that are responsible for ingesting data (those that create the FlowFile in NiFi) should be configured for "Primary node" only operation. Al processors within the body of a dataflow (any processor that accepts an inbound connection to it) should be configured to always run on all nodes in your cluster.
-
If I had to guess one or more of the following is occurring for you:
1. Processors within the body of your dataflows are configured with "Primary node". This means that any FlowFiles ingested on old primary node will end up queued in front of one of these now primary node only configured processors that is no longer getting scheduled on that old primary node.
2. Your primary node only processors have some configuration that is dependent on a local file that that is not present on every node in same directory with same permissions. (for example, Using a private key in listSFTP and that private key was not placed on all nodes).
-
Thank you,
Matt
-
If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link.
