Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
Contributor

Introduction into Process Group Variables

The ability configure variables at runtime in the UI was introduced in Apache NiFi 1.4 and will subsequently land in HDF 3.1. Variables can be referenced in property values of Processors and Controller Services that support expression language. This provides a centralized place to configure common values that may be referenced by multiple components and values that may be environmentally dependent. This guide provides a brief introduction to their usage.

Configuration

Variables are configured in a dialog that is accessed through the context menu.

46406-screen-shot-2017-12-21-at-51046-pm.png

The menu item is available when the selection is a single Process Group. Below is an example of what the variable dialog would look like.

46407-screen-shot-2017-12-21-at-51103-pm.png

The table will list all variables that are available in the scope of the current Process Group, the variable name, and variable value. Only variables defined in the current Process Group are editable. The table also contains a navigation link to go to the Process Group where inherited variables are defined. On the right side of the dialog, the variable selected in the table is shown along with all components that reference it.

Scoping

Processors and Controller Services encapsulated in the current Process Group will have access to variables defined at that Process Group and all ancestor Process Groups. If a nested Process Group defines a variable with the same name as one defined at a higher scope, it will override that value. This is visualized in the Variable dialog using a strikethrough. This overriding process can be summarized by saying that when a component references a variable, the value will come from the closest definition.

46408-screen-shot-2017-12-21-at-51119-pm.png

Additionally, variables defined in the properties files specified using the nifi.variable.registry.properties configuration can be overridden by Process Group Variables.

Referencing

Referencing a variable is done using expression language. If a Processor or Controller Service property supports expression language, a variable name can be used as the subject.

46409-screen-shot-2017-12-21-at-51149-pm.png

The order of precedence for expression language subjects is flow file attributes, Process Group variables, variables defined in the properties file, system environment variables, and finally JVM system property. For more information regarding Expression Language, please refer to the Expression Language Guide available in Apache NiFi.


screen-shot-2017-12-21-at-51046-pm.png
9,643 Views
Comments
New Contributor

Hello,

i would like to pass to ListSFTP processor some variables from DB but as far i know, this processor does not accept any flowfile as input. Is there a way to override processor group variables?

Other solutions are accepted aswell.

Thanks

Not applicable

The variables present in processGroup are not exported into templates, which i would like to use in minifi.

Contributor

@Jayanth S Are the variable's your templated components referencing defined in a Process Group that is also part of this template?

Not applicable

Yes. One of the processor inside processor group is using that variable.

The variable is not exported but the value of the property is showing as ${'var'}. I am able to change the value using env variable.

Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 09:26 AM
Updated by:
 
Contributors
Top Kudoed Authors