In Nifi, when a node in NiFi cluster goes down, the complete flow on my canvas goes into read-only mode, i.e. I won't be able to change the state of any processor until I remove the failed node through the NiFi cluster UI.
In our scenario, we are continuously getting data from the ConsumeKafka processor into our flow and at the same time performing reconciliation of records that we receive in a later stage of the flow. So if anyone of the node suddenly goes down, the reconciliation logic will throw an error because of wrong record mismatch, as we are losing the flow files in the node that went down and won't be able to stop the logic for it either until the node is not disconnected.
How do we prevent such a scenario? Is there a way where we can internally check the connection of the nodes every time we do a reconciliation? Or any other workaround to stop the reconciliation until the failed node is disconnected/reconnected?