Skip to main content
ChuckE
ChuckE
Gold ⭐️
May 22, 2023
Question

Referencing Parameter Contexts from different scopes

Hello NiFi Community,

 

I have an AvroSchemaRegistry controller service and it references a bunch of schemas stored in a parameter contexts variables.  Both the AvroSchemaRegistry and Parameter Contexts are at the same scope level. However, I have a very large number of schemas so for organizational purposes I'd like to organize them into separate Parameter Contexts: red_schemas, green_schemas,  etc...

But I'm still using just the one AvroSchemaRegistry controller service.  

 

Is it possible to create child Parameter Contexts (red, green, etc...) that inherit from the main Parameter Context and then reference the variables from a single controller service, which is at a higher scope?  If so, what is the syntax for referencing variables in a different scope than the controller service?

 

Thanks for reading.

2 replies

MattWho
Titanium ⭐️⭐️⭐️
MattWho
Titanium ⭐️⭐️⭐️
May 30, 2023

@ChuckE 

I think what you are looking for is described in this in progress Apache NiFi Jira:
https://issues.apache.org/jira/browse/NIFI-8487

However, creating a new parameter-context that can Inherit from other parameter-contexts is possible as of Apache NiFi 1.15. 
https://issues.apache.org/jira/browse/NIFI-8490

It was then broken in Apache NiFi 1.16 and fixed again in 1.16.3 and 1.17.0+
https://issues.apache.org/jira/browse/NIFI-10096

 

From the NiFi Global menu in the upper right corner, you can select "Parameter Contexts".  Use the "+" icon to add you various parameter contexts to your NiFi.

1. Add a new Process Group (PG) to the canvas. 

2. Right click on that PG and select "configure" from displayed context menu.

3. Under "Process Group Parameter Context", select option to create new parameter context. 
4. Give new parameter-context a name in the "Settings" tab and click apply.
5. Right click on PG and select "parameters" from displayed context menu.

6. Select the "inheritance" tab.  All available parameter context for which your user is authorized to access will be displayed to the left.  Start dragging and dropping the parameter context from which your want your new parameter context to inherit from to the right side. If multiple parameters with same name exist, the one from the parameter context furthest up the list on the right side will have it's value used.  If the new parameter context has its own defined parameters, they take precedence over those of same name in the inherited parameter contexts.
7. Click Apply.

MattWho_0-1685477778508.png

You now have access to unique parameters from all the inherited Parameter Contexts.  
When configuring a parameter context in a component property, type "#{" and then  hold "control key" while hitting "spacebar" to show full list of selectable parameters from all inherited parameter contexts.

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt

Translate
ChuckE
ChuckEAuthor
Gold ⭐️
May 31, 2023

Thanks for taking the time to reply Matt, but this doesn't address my question.  Let me use an example and hopefully that will clarify what I'm trying to do.

 

Within a process group named "transit", there is a "Process Group Parameter Context" named "transit_variables".   This parameter context (transit_variables) includes all the high level variables used throughout the process group, but it also contains a large number of Avro schemas which are referenced in an AvroSchemaRegistry controller service, which is defined within the main process group.  

 

For the sake of organizational purposes I'd like to move these Avro schemas into their own parameter context, BUT (this is the catch), I want access to them from the AvroSchemaRegistry whose scope is associated with the process group, whose scope is set to the parent parameter context.

 

I can "solve" this problem by creating a child process group (e.g. transit_data_parser) with a new child parameter context (e.g. transit_schemas), then add additional AvroSchemaRegistry, JSONTreeReader, JSONRecordsetWriter controller services, which are coupled with the child parameter context "transit_schemas".  The process group would contain a single processor with an input port and and output port, and would be used solely for the purpose of reading a flowfile into a NiFi Recordset object with Avro validation.   

 

This so-called solution is very tightly coupled and as such not very flexible.  It means that every time I want to use a schema I need to create one of these tightly couple process groups, which defeats the whole purpose of the context parameter concept.

 

What would be better is if I could reference these child schemas using a hierarchal object syntax, e.g. 

#{transit_variables.transit_schemas}.  Using this syntax from within a single AvroSchemaRegistry controller service I can reference parameters at the parent level and all parameters at the child levels. 

Translate
Terms & ConditionsAccessibility statement