This article assumes that you're already configured to use Ranger authorization in HDF 3.0.x or earlier and that Ranger instance is syncing the available users/groups with your Directory Server. In order to leverage policies defined with these groups, we'll need to upgrade to HDF 3.1.0 or later and migrate your configuration to utilize a new Ranger authorizer in NiFi. In short, you’ll need to update your authorizers.xml file to configure a userGroupProvider and modify your authorizer to be a ManagedRangerAuthorizer. The idea is that previously an Authorizer was configured which was responsible for making access decisions, managing policies, and managing users/groups. Whether or not the Authorizer supported managing policies and users/groups was optional. The default File base authorizer supported this. The existing Ranger authorizer did not (it only made access decisions). For HDF 3.1 we decoupled these concepts so you can independently configure the Authorizer (makes access decisions), an AccessPolicyProvider (manage policies), and a UserGroupProvider (manage users/groups). Documentation exists for these three concepts and includes examples [1]. What you’ll need to do is update your authorizers.xml and add a UserGroupProvider to bind your users/groups from your Directory Server that Ranger is syncing to. There are a couple detailed examples of binding users/groups from a Directory Server using the LdapUserGroupProvider. Searching for this term in [1] should direct you to the relevant examples. Following the concepts defined above, the ManagedRangerAuthorizer will act as the AccessPolicyProvider (since the policies are defined there) and the Authorizer (Ranger ultimately makes the access decisions). To configure your ManagedRangerAuthorizer, we can take your existing configuration for RangerNiFiAuthorizer and 1) modify the class and 2) add a reference to your LdapUserGroupProvider. The remainder of the configuration should remain unchanged. This should look something like this: <authorizer> <identifier>ranger-provider</identifier> <class>org.apache.nifi.ranger.authorization.ManagedRangerAuthorizer</class> <!-- 1) UPDATE CLASS NAME --> <property name="User Group Provider">ldap-user-group-provider</property> <!-- 2) REFERENCE USER GROUP PROVIDER --> <property name="Ranger Audit Config Path">…</property> <property name="Ranger Security Config Path">…</property> <property name="Ranger Service Type">…</property> <property name="Ranger Application Id">…</property> <property name="Ranger Admin Identity">…</property> </authorizer> This is all that you should need to do. In totality, the structure of your authorizers.xml file should looks like this: <authorizers> <userGroupProvider> <identifier>ldap-user-group-provider</identifier> <class>org.apache.nifi.ldap.tenants.LdapUserGroupProvider</class> <property name="Authentication Strategy">...</property> <property name="Manager DN">...</property> <property name="Manager Password">...</property> <property name="TLS - Keystore">...</property> <property name="TLS - Keystore Password">...</property> <property name="TLS - Keystore Type">...</property> <property name="TLS - Truststore">...</property> <property name="TLS - Truststore Password">...</property> <property name="TLS - Truststore Type">...</property> <property name="TLS - Client Auth">...</property> <property name="TLS - Protocol">...</property> <property name="TLS - Shutdown Gracefully">...</property> <property name="Referral Strategy">...</property> <property name="Connect Timeout">...</property> <property name="Read Timeout">...</property> <property name="Url">...</property> <property name="Page Size">...</property> <property name="Sync Interval">...</property> <property name="User Search Base">...</property> <property name="User Object Class">...</property> <property name="User Search Scope">...</property> <property name="User Search Filter">...</property> <property name="User Identity Attribute">...</property> <property name="User Group Name Attribute">...</property> <property name="Group Search Base">...</property> <property name="Group Object Class">...</property> <property name="Group Search Scope">...</property> <property name="Group Search Filter">...</property> <property name="Group Name Attribute">...</property> <property name="Group Member Attribute">...</property> </userGroupProvider> <authorizer> <identifier>ranger-provider</identifier> <class>org.apache.nifi.ranger.authorization.ManagedRangerAuthorizer</class> <property name="User Group Provider">ldap-user-group-provider</property> <property name="Ranger Audit Config Path">...</property> <property name="Ranger Security Config Path">...</property> <property name="Ranger Service Type">...</property> <property name="Ranger Application Id">...</property> <property name="Ranger Admin Identity">...</property> </authorizer> </authorizers> [1] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#authorizers-setup ... View more

Introduction into Process Group Variables The ability configure variables at runtime in the UI was introduced in Apache NiFi 1.4 and will subsequently land in HDF 3.1. Variables can be referenced in property values of Processors and Controller Services that support expression language. This provides a centralized place to configure common values that may be referenced by multiple components and values that may be environmentally dependent. This guide provides a brief introduction to their usage. Configuration Variables are configured in a dialog that is accessed through the context menu. The menu item is available when the selection is a single Process Group. Below is an example of what the variable dialog would look like. The table will list all variables that are available in the scope of the current Process Group, the variable name, and variable value. Only variables defined in the current Process Group are editable. The table also contains a navigation link to go to the Process Group where inherited variables are defined. On the right side of the dialog, the variable selected in the table is shown along with all components that reference it. Scoping Processors and Controller Services encapsulated in the current Process Group will have access to variables defined at that Process Group and all ancestor Process Groups. If a nested Process Group defines a variable with the same name as one defined at a higher scope, it will override that value. This is visualized in the Variable dialog using a strikethrough. This overriding process can be summarized by saying that when a component references a variable, the value will come from the closest definition. Additionally, variables defined in the properties files specified using the nifi.variable.registry.properties configuration can be overridden by Process Group Variables. Referencing Referencing a variable is done using expression language. If a Processor or Controller Service property supports expression language, a variable name can be used as the subject. The order of precedence for expression language subjects is flow file attributes, Process Group variables, variables defined in the properties file, system environment variables, and finally JVM system property. For more information regarding Expression Language, please refer to the Expression Language Guide available in Apache NiFi. ... View more

When running Apache NiFi behind a proxy there are a couple of key items to follow during deployment. 1) NiFi is comprised of a number of web applications (web ui, web api, documentation, custom ui's, data viewers, etc). So the mapping needs to be configured for the root path. That way all context paths are passed through accordingly. For instance, if only the /nifi context path was mapped, the custom ui for UpdateAttribute will not work since it's available at /update-attribute-ui-<version>. 2) NiFi's REST API will generate URI's for each component on the graph. Since requests are coming through a proxy, certain elements of the URI's being generated need to be overridden. Without overriding the users will be able to view the dataflow on the canvas but will be unable to in modify existing components. Requests will be attempting to call back directly to NiFi, not through the proxy. The elements of the URI can be overridden by adding the following HTTP headers when the proxy generates the HTTP request to the NiFi instance: X-ProxyScheme - the scheme to use to connect to the proxy X-ProxyHost - the host of the proxy X-ProxyPort - the port the proxy is listening on X-ProxyContextPath - the path configured to map to the NiFi instance 3) If NiFi is running securely, any proxy needs to be authorized to proxy user requests. These can be configured in the NiFi UI through the Global Menu. Once these permissions are in place proxies can begin proxying user request. The end user identity must be relayed in a HTTP header. For example, if the end user sent a request to the proxy, the proxy must authenticate the user. Following this the proxy can send the request to NiFi. In this request an HTTP header should be added as follows. X-ProxiedEntitiesChain: <end-user> If the proxy is configured to send to another proxy, the request to NiFi from the second proxy should contain a header as follows. X-ProxiedEntitiesChain: <end-user-identity><proxy-1-identity> ... View more

In Apache NiFi, Controller Services are shared services that can be used by Processors, Reporting Tasks, and other Controller Services. In order to modify a Controller Service, all referencing components must be stopped. This means that the user attempting to modify the Controller Service, must also have permissions to modify all referencing components. With the introduction of multi-tenant authorization, Controller Service scoping was updated to ensure service usage does not become so broad that it becomes impossible to update the Controller Service because no users are allowed to modified all components that reference it. Within the dataflow, to scope a Controller Service and limit it's availability, a Controller Service can be created within any Process Group. Controller Services created within a Process Group will be available to all descendent components. This means that the service will be available to all Processors and Controller Services defined in that Process Group and below. In a typical multi-tenant environment users permissions will typically be bound to a Process Group. By scoping the Controller Service in the same manner, it's usage is limited accordingly. Within the Controller (the infrastructure that executes the dataflow), Controller Service availability is limited to Reporting Tasks and other services defined here. This scoping works in hand with the scoping implemented within the dataflow context. Users with permissions to Reporting Tasks will also have access to services defined here. Realizing these differences in Controller Service availability was often difficult to understand. However, upcoming UX changes to Controller Services helps alleviate some of these challenges. Below are screenshots for the different contexts and highlight the changes which help understand the service availability. ... View more