Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to manage node failure in Nifi Cluster ?

How to manage node failure in Nifi Cluster ?

New Contributor

Hi,

In Nifi, when a node in NiFi cluster goes down, the complete flow on my canvas goes into read-only mode, i.e. I won't be able to change the state of any processor until I remove the failed node through the NiFi cluster UI.

In our scenario, we are continuously getting data from the ConsumeKafka processor into our flow and at the same time performing reconciliation of records that we receive in a later stage of the flow. So if anyone of the node suddenly goes down, the reconciliation logic will throw an error because of wrong record mismatch, as we are losing the flow files in the node that went down and won't be able to stop the logic for it either until the node is not disconnected.

How do we prevent such a scenario?
Is there a way where we can internally check the connection of the nodes every time we do a reconciliation? Or any other workaround to stop the reconciliation until the failed node is disconnected/reconnected?

Don't have an account?
Coming from Hortonworks? Activate your account here