question Re: Unable to import templates in Nifi in Support Questions

Unable to import templates in Nifi

broobalaji — Tue, 12 Mar 2024 06:54:30 GMT

Hi Team,
We have a usescase wherein we want to import the templates directly into nifi canvas using the rest apis provided by nifi we are hitting with the below error. If anyone faced this issue previously please assisst.

we are able to upload, download but import is not working

To get the token

bearer=$(curl 'https://nifi-host:nifi-port/nifi-api/access/token' --data 'username=username&password=password' --compressed --insecure)

To import the template

curl -i -X -k POST -H "Content-Type: application/xml" -H "Authorization: Bearer $bearer" -F "template=@/root/tmp/template.xml" https://nifi-host:nifi-port/nifi-api/process-groups/processor-group-id/templates/import

Nifi version - 1.8.0.3.3.1.0-10

error - HTTP 400 Bad Request

Thanks

Re: Unable to import templates in Nifi

VidyaSargur — Tue, 12 Mar 2024 09:05:57 GMT

@broobalaji, Welcome to our community! To help you get the best possible answer, I have tagged in our NiFi experts @MattWho @SAMSAL @TimothySpann @steven-matison @ckumar who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

Re: Unable to import templates in Nifi

ckumar — Tue, 12 Mar 2024 09:22:54 GMT

If you have to load the new template from the filesystem/local to the NiFi cluster, the following API call should be made with POST method

nifi-api/process-groups/{pg-id}/templates/upload

Thank you

Re: Unable to import templates in Nifi

broobalaji — Tue, 12 Mar 2024 10:12:44 GMT

Hi @ckumar
Is there any way (api call) to directly import the templates into the nifi canvas

/process-groups/{id}/templates/import

we got this api call in the nifi api doc

doc - https://nifi.apache.org/docs/nifi-docs/rest-api/index.html

This is not working for us

Thanks

Re: Unable to import templates in Nifi

ckumar — Tue, 12 Mar 2024 10:50:19 GMT

The first step is to create a template, which can be created within in NiFi cluster from another flow or uploaded from the filesystem, this action will load the templates under

/nifi-api/flow/templates

Then use the below one to create a process group from a template.
/process-groups/{id}/template-instance

import is not the right API call, you can verify this from from web developer tool doing the same actions from UI.

Re: Unable to import templates in Nifi

broobalaji — Tue, 12 Mar 2024 11:43:34 GMT

Thanks we were able to import the templates using the api calls

curl -X POST -H "Content-Type: multipart/form-data" -H "Authorization: Bearer $bearer" -F "template=@/root/api_testing/template.xml" https://host-ip:host-port/nifi-api/process-groups/process-group-id/templates/upload -k

First we need to upload the template from local as it generates a id for the template which we can use for importing the template into the canvas

For importing the template

curl -i -X POST -H "Content-Type: application/json" -H "Authorization: Bearer $bearer" -d '{"templateId":"template-id","originX":100,"originY":100}' https://host-ip:host-port/nifi-api/process-groups/process-group-id/template-instance -k

Re: Unable to import templates in Nifi

MattWho — Tue, 12 Mar 2024 13:22:25 GMT

@broobalaji

HDF 1.8.0.3.3.1.0-10 was released way back in 2017. I strongly recommend upgrading to a much newer release of CFM.

NiFi Templates have been deprecated and are completely removed as of Apache NiFi 2.x releases. Apache NiFi deprecated templates for a number of reasons:

1. Templates uploaded to NiFi (even if not instantiated/imported to the NiFi canvas reside within NiFi's heap memory space)
2. Large uploaded templates or many uploaded templates can have a substantial impact on NiFi performance because of the amount of heap they can consume. Simply increasing the size of NiFi's heap is also not the best solution to that heap usage as large heaps just lend themselves to longer stop-the-world garbage collections with the JVM.
3. Apache NiFi deprecated and moved away from using xml based flow in favor of json flow definitions around the Apache NiFi 1.16 time frame.

Flow definitions (JSON files) can exported and imported without uploading them in to heap memory within NiFi.

The above info aside....
It is best to use the developer tools available in your web browser to inspect/capture the rest-api call being made when you perform the same steps directly via the NiFi UI. This makes it easy to understand the calls that need to be made in your automation. I also encourage you if you continue to use templates to upload, import to UI, and then delete the uploaded template to minimize heap impact.

Thanks,
Matt