Backup Of Nifi Cluster with 3 Nodes

@jame1997 

1. Everything that is exists on the canvas resides in heap and is persisted to the flow.json.gz file written to the NiFi conf directory (default).  With every change made within the NiFi UI, the current flow.json.gz if moved to an archive directory and new flow.json.gz is generated.  All Nodes in a NiFi cluster use the same flow.json.gz.  You don;t need to stop NiFi to make a copy of the flow.json.gz file.  The NiFi flow.json.gz will contain encrypted values for all sensitive properties entered in any component on the NiFi UI canvas. In order for a NiFi instance to load a flow.json.gz, it must be cnfigured with the same "nifi.senstive.props.key" password used by the NiFi where the flow.json originated.  So always a good idea to backup your NiFi's conf directory (especially the nifi.properties file).  Keep in mind that all nodes in the NiFi Cluster must have same sensitive props key password configured, so unless you lose your entire NiFi cluster, you can get the flow.json.gz an sensitive.props.key form any nodes still accessible.

2. I am not crystal clear on what "backup all the processes" means.  Are you referring to all the dataflows built on the NiFi canvas?

Some components (processors, controller services, reporting tasks) may have dependencies on other things like local/cluster state, local files, etc.  So those items need to be taken into consideration with any backup planning.  Each node in a NiFi cluster has a local state directory which hold state for components that use local state (listFile for example).  For other components, cluster state is used (cluster state is written to Zookeeper and Zookeeper quorum maintains protection of that information).  State is always changing as your dataflow is running, so it is not something to easily backup specifically for local state that has not redundancy across NiFi nodes. 

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Hello Matt,
Thanks for the information.  With the flow.json.gz if a process group get deleted then the entire fflow.json.gz need to be restore.  I was wondering if there is another way to backup the running progress groups without taking a complete template backup that I can backup that will list the individual process groups so I can choose the one that's require a restore instead of restore all the process groups. 

Thanks!