Automate NIFI flow and start processor group automatically

[#1]

Hi  , 

I want to automate my NiFi flow deployment process.
Currently, I have a NiFi template generator that works with multiple XML and configuration files where I define properties like Kafka topic names, broker IPs, and other NiFi-related settings. Using this template generator, I parse an existing sample NiFi XML template and generate a new XML file with all the required properties populated.

Right now, I upload this generated XML file manually through the NiFi UI. After that, I manually create a new process group on the NiFi canvas, import the template into this group, and then start the processor group and all required controller services.

I want to automate this entire flow—right from generating the template and creating the process group, to uploading the template into NiFi and starting the processors and controller services—so that no manual steps are needed in the NiFi UI.

[#2]

@Rohit1997jio 

I would discourage you from continuing to any automation using Apache NiFi 1.x "templates".  NiFi templates were deprecated in the now end of life Apache NiFi 1.x major release version.  They were officially removed in the Apache NiFi 2.x major release. versions.

Uploaded NiFi templates consume valuable NiFi heap memory.  The replacement is NiFi Flow Definitions.   Apache NiFi started moving away from XML based configuration files in around Apache NiFi 1.16.   While all NiFi 1.x retained the flow.xml.gz until its final release version, the flow.json.gz was introduced in 1.16 and is what is loaded on NiFi startup.  

The only difference is the deprecated flow templates allowed you to create a template that consists of just individual low level components, while flow definitions can be created at only the "Process Group" level.

Flow Definitions can be created by downloaded the flow definition directly from a process group. 

Deploying Flow Definitions is also easier.  Simply drag the add new "Process group" icon to canvas and click on the "Browse" icon in the Process Group Name field box.  You will be prompted to provide the downloaded flow definition json file.

This upload can also be done via a rest-api call. The Apache NiFi 1.x major release REST API covers the api calls to do everything that you can accomplish via directly in the NiFI UI. But you can also use your browser build in "Developer Tools" capture these rest-api calls as you perform the actions directly in the NiFi UI.   The Developer tools even give you an option to "Copy as curl" making it even easier to learn (browser will add many unnecessary headers you can opt to remove.  If you remove a header that is needed, the curl command will fail): 

NOTE: Apache NiFi 1.x is end of life and no longer being contributed to.  This means no more bug and CVE issues being addressed in the the 1,x release line. The Apache NiFi 2.x major release is the new supported major release (where templates no longer exist).

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt