question Re: Nifi: Using parameters in Support Questions

Nifi: Using parameters

Jarinek — Sun, 07 Feb 2021 19:55:57 GMT

Hello,

I have a question regarding the concept of parameter context.

The admin guide says : A Process Group can only be assigned one Parameter Context, while a given Parameter Context can be assigned to multiple Process Groups.

OK, let's assume you have internal kafka or database, and some external system you connect to. You might want to have a parameter "group"
for the internal kafka connection, and another parameter contexts for each external system connection.

However, your flow, clearly a part a process group, reads data from external system and stores data internally. Thus you need actually 2 connections in the flow,
it means 2 parameter contexts, which is not possible. To solve that, you assign the connections (controller services) to the parent group and the parameters.

Finally, for connections which are common to many flows, like your kafka, databases, and HttpSSL you land in the parent NiFi Flow, having parameter context which contains
a bunch of pretty unrelated parameters (for kafka, databases, ..).

What is see, there is hardly a possibility to reuse parameter context
as they cannot be combined and thus you have 1:1 relationship between par. context and group just like for variables assigned to a processor group.

Did I miss something? What is actually the big advantage of parameters over variables?

Best regards
Jaro

Re: Nifi: Using parameters

TimothySpann — Mon, 08 Feb 2021 15:52:44 GMT

Variables are deprecated. Parameters are new and easy to externalize. I can use them in devops processes via REST, NiFi CLI and Python. You can programatically build parameter contexts and parameters and assign them to process groups.

Parameters are getting some upgrades to do some of the more advanced things you mentioned.

https://www.datainmotion.dev/2020/09/devops-working-with-parameter-contexts.html

And they help power stateless nifi.,

https://www.datainmotion.dev/2019/11/exploring-apache-nifi-110-parameters.html

https://www.datainmotion.dev/2021/01/automating-starting-services-in-apache.html

Re: Nifi: Using parameters

MattWho — Mon, 08 Feb 2021 16:20:03 GMT

@Jarinek

NiFi Variables can only be used by component properties that support NiFi's Expression Language (EL).

NiFI Parameters can be used in ANY component property including those that are encrypted. This gives more flexility to users, especially those users who use NiFi-Registry to promote version controlled Process groups across multiple NiFi instances/clusters. It is often the case that different environments have different URLs and passwords in use within the same dataflows. A Dataflwo can thus be promoted to another environment that simply uses different Parameter values thus not requiring the user to update a large number of components each time a new version of flow is promoted from one environment to another.

You are correct that Parameters are similar to Variables in respect to assignment to a process group. You can only have one Parameter context assign to a Process Group.

Hope this helps,
Matt

Re: Nifi: Using parameters

Rex13 — Fri, 03 Sep 2021 11:20:01 GMT

From what I can see, the Parameters will also be versioned by NiFiRegisty. So same URL/service account will be promoted to other environment.

Is there any way to keep different Parameter in different env?

Re: Nifi: Using parameters

MattWho — Wed, 08 Sep 2021 18:40:25 GMT

@Rex13

Parameters are included when you version control a PG to the NiFi-Registry. And if you then import that version controlled PG to another NiFi, that parameter context will be imported as well.

Once imported the parameter context can be changed or updated as needed in the new environment. Later if you commit version 2 of that same PG to NiFi-Registry and then upgrade to version 2 in the second environment, the parameter context changes you made locally on second environment will remain and will not be overwritten.

If you found this response addressed your query, please take a moment to login and click on "Accept as Solution" below any solution that helped.

Thank you,

Matt

Re: Nifi: Using parameters

Rex13 — Thu, 09 Sep 2021 00:56:11 GMT

Thank you @MattWho for your kindly reply.

We're using Git repo as the storage for NiFi-Registry.

And different environment has different Git branch to avoid any changes which are not expected.

For example, we will merge the code from test branch to prod branch after everything have been tested.

Just as what you said, it won't have any impact if I don't change any parameters in version 2.

But it will cause conflict if some parameters are also changed.

Re: Nifi: Using parameters

Rex13 — Wed, 22 Sep 2021 02:41:11 GMT

Thank you @MattWho

I've tried API and it won't change the parameter which has already been imported before.

I didn't found "Accept as Solution". But I clicked the "KUDO" button. Thanks.

Re: Nifi: Using parameters

rcx — Thu, 18 Nov 2021 01:21:33 GMT

if possible dynamic set parameter,