Upload multiple nifi templates at once

syntax_ — Tue, 27 Apr 2021 11:22:14 GMT

How do I upload multiple nifi templates at once?Nifi only allows to upload one single template/xml file at a time. How do I upload all the templates in bulk?

Re: Upload multiple nifi templates at once

MattWho — Thu, 06 May 2021 21:01:07 GMT

@syntax_

NiFi does not provide a method for uploading templates in bulk. But anything you can do via the UI, you can also do via rest-api calls [1] through curl. So you could script the bulk upload through rest-api external to NiFi's UI.

I would strongly discourage bulk uploading templates to NiFi. Templates should be uploaded as needed, instantiate to the canvas and then template deleted from NiFi. All uploaded templates even if not instantiated to the canvas become part of the flow.xml loaded in to heap memory. So keeping a large number of templates uploaded to your NiFi can have a considerable impact on JVM heap memory usage because now you have not only your active flow on the canvas in memory, but also these templates.

Additionally, In a NiFi cluster on startup the flow.xml.gz is uncompressed, loaded in to heap memory, and a flow fingerprint created on every node. These flow fingerprints are then compared to make sure all nodes joining the cluster are running with the same flow.xml. Since these templates can just add unnecessary size to the flow.xml.gz, this can impact startup times and flow fingerprint comparison time.

A better approach is to get your reusable flow migrated into NiFi-Registry [2]. You can have 1 to many NiFi's all connect to a single NiFi-Registry where all your flows exist. Then all you need to do is drag a process group to the canvas and select "import" which will allow you to select one of these flows from NiFi-Registry and add it to your canvas. Now you have your flow without the additional impact on heap from having it in NiFi both as a template and as a dataflow on your canvas with easy access to load same flow over and over. I encourage you to explore NiFi-Registry.

[1] http://nifi.apache.org/docs/nifi-docs/rest-api/index.html

[2] https://nifi.apache.org/registry.html

If you found this response helped with your query, please take a moment to login in and click accept on this solution.

Thanks,

Matt

question Upload multiple nifi templates at once in Support Questions

Upload multiple nifi templates at once

Re: Upload multiple nifi templates at once