Created 04-02-2024 08:53 AM
I'm just curious to know. When I start a processor in the NIFI cluster, where does NIFI store the processor state "RUNNING" in this case?
Created 04-02-2024 01:22 PM
@edim2525
The entire datflow(s) reside in NiFi heap memory and are persisted to disk in the flow.json.gz file. This includes the current set state for each component. Every time a change is made to the NiFi canvas, the current persisted flow.json.gz is moved to archive and a new flow.json.gz is written.
When NiFi is started it will load the flow.json.gz in to NiFi heap memory and set the state to the state for each component recorded in the the flow.json.gz that is loaded. Only time this is not true is when the nifi.properties property "nifi.flowcontroller.autoResumeState" has been set to false. When set to false, all "RUNNING" components set in the flow.josn.gz will load into heap as "STOPPED". NiFi will then archive the current flow.json.gz and write a new flow.json.gz with those new "STOPPED" states.
Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.
Thank you,
Matt
Created 04-02-2024 01:22 PM
@edim2525
The entire datflow(s) reside in NiFi heap memory and are persisted to disk in the flow.json.gz file. This includes the current set state for each component. Every time a change is made to the NiFi canvas, the current persisted flow.json.gz is moved to archive and a new flow.json.gz is written.
When NiFi is started it will load the flow.json.gz in to NiFi heap memory and set the state to the state for each component recorded in the the flow.json.gz that is loaded. Only time this is not true is when the nifi.properties property "nifi.flowcontroller.autoResumeState" has been set to false. When set to false, all "RUNNING" components set in the flow.josn.gz will load into heap as "STOPPED". NiFi will then archive the current flow.json.gz and write a new flow.json.gz with those new "STOPPED" states.
Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.
Thank you,
Matt
Created 04-03-2024 05:06 AM
Thank you for the detailed explanation, Matt