How to access your secured NiFi instance or cluster:
So you have secured your NiFi instance (meaning you have configured it for https access) and now you are trying to access the https web UI. Once NiFi is secured, any entity interacting with the UI will need to successfully authenticate and then be authorized to access the particular NiFi resource(s). As of HDF 2.1.1 or Apache NiFi 1.1.0, NiFi supports authentication via user certificates (default - always enabled), Kerberos/Spnego, or username and password based authentication via LDAP, LDAPS, or kerberos. The intent of this article is not to cover the authentication process, but rather to cover the initial admin authorization process. We assume for this article that authentication is successful. How do you know? A quick look in the nifi-user.log will tell you if your users authentication was successful.
Following successful authentication comes NiFi Authorization. NiFi authorization can be handled by NiFi's default built in file based authorizer or handled externally via Ranger. This article will cover the default built in file based authorizer.
NiFi's built in file based authorization:
There are four files in NiFi that contain properties used by NiFi file based authorizer:
- nifi.properties
- authorizers.xml
- users.xml
- authorizations.xml
We will start by showing what role each of these files plays in NiFi user/server authorization.
nifi.properties file (Pattern Mapping):
The nifi.properties file a lot of key/value pairs that are used my NiFi's core. This file happens to be where users can define identity mapping patterns. These properties allow normalizing user identities such that identities coming from different identity providers (certificates, LDAP, Kerberos) can be treated the same internally in NiFi. It is the resulting value from a matching pattern that is passed to the configured authorizer (NiFi's file based or Ranger). NiFi includes two examples that are commented out in the nifi.properties file; however, you can add as many unique identity mapping patterns as you need.
nifi.security.identity.mapping.pattern.dn=^CN=(.*?),OU=(.*?),O=(.*?),L=(.*?),ST=(.*?),C=(.*?)$ nifi.security.identity.mapping.value.dn=$1@$2 nifi.security.identity.mapping.pattern.kerb=^(.*?)/instance@(.*?)$ nifi.security.identity.mapping.value.kerb=$1@$2nifi.security.identity.mapping.value.kerb=$1@$2
All mapping patterns use java regular expressions. They are case sensitive and white space matters between elements.
for example ^CN=(.*?),OU=(.*?),O=(.*?),L=(.*?),ST=(.*?),C=(.*?)$ would match on:
CN=John Doe,O=SME,L=Bmore,ST=MD,C=US
but would not match on:
cn=John Doe, o=SME, l=Bmore, st=MD, c=US (Note the lowercase and white spaces)
Assuming a DN of CN=John Doe,O=SME,L=Bmore,ST=MD,C=US
the associated mapping value would return John Doe@SME
Additional mapping patterns can be added simply by adding additional properties to the nifi.properties file similar to the above examples except each must have a unique value following nifi.security.identity.mapping.pattern. or nifi.security.identity.mapping.value. . For example:
nifi.security.identity.mapping.pattern.dn2=^CN=(.*?), OU=(.*?)$ nifi.security.identity.mapping.value.dn2=$1
While you can create as many mapping patterns as you like, it is important to make sure that you do not have more then one pattern that can match your incoming user/server identity. Those user identities are run against every configured pattern and only the last pattern that matches will be applied.
authorizers.xml (Default configuration supports file-provider)
This file is where you will setup your NiFi file based authorizer.
It is this file in which you will find the "Initial Admin Identity" property. It is very important that you correctly define an "Initial Admin Identity" before starting your secured https NiFi for the first time. (no worries if you have not, I will discuss how to fix issues when you did not or had a typo).
If you are securing a NiFi cluster, you will also need to configure a "Node Identity x" for each node in your cluster (where "x" is sequential numbers). *** Don't forget to remove the comment lines "<!---" and "-->" from around these properties.
So, what values should I be providing to these properties? That depends on a few factors:
- Which authentication method did I use?
- User/server/node certificates (default) - User certificates will have a DN in the certificate for that user. This full DN is evaluated by any configured identity mapping patterns and the result is passed to the authorizer.
- NiFi nodes can only use server certificates to authenticate. Each server is issued server certificates and the Full DNs form those certificates are evaluated by any configured identity mapping patterns and the result is passed to the authorizer.
- Kerberos/Spnego - The users principal is evaluated by any configured identity mapping patterns and the result is passed to the authorizer.
- LDAP/LDAPS - Users are presented with a login screen. NiFi's LDAP configuration can be setup to pass either the DN returned by LDAP for the user (default) or the username (supplied at login screen). This return is evaluated by any configured identity mapping patterns and the result is passed to the authorizer.
- Kerberos - Users are presented with a login screen. The user's principal is evaluated by any configured identity mapping patterns and the result is passed to the authorizer.
- User/server/node certificates (default) - User certificates will have a DN in the certificate for that user. This full DN is evaluated by any configured identity mapping patterns and the result is passed to the authorizer.
- Did I setup identity pattern mappings?
- If no identity mapping patterns were defined, the full return from the configured authentication is passed to the authorizer.
- If the user/server identity fails to match on any of the defined identity mapping patterns, the full return from the configured authentication is passed to the authorizer.
What ever the final resulting value will be is what needs to be entered in the "Initial Admin Identity" and " Node Identity x" properties:
Let's assume the following user/server DNs and that multiple identity mappings were setup in the nifi.properties file:
| Sample entity DN: | Configured Identity Mapping Pattern: | Configured Identity Mapping Value: | Resulting value: |
| cn=JohnDoe,ou=SME,dc=work | ^cn=(.*?),ou=(.*?),dc=(.*?),dc=(.*?)$ | $1 | JohnDoe |
| CN=nifi-server1, OU=NIFI | ^CN=(.*?), OU=(.*?)$ | $1 | nifi-server1 |
| CN=nifi-server2, OU=NIFI | ^CN=(.*?), OU=(.*?)$ | $1 | nifi-server2 |
Your authorizers.xml file would then look like this:
The values configured here will be used to seed the users.xml and authorizations.xml files.
users.xml
The users.xml file is produced the first time and only the first time NiFi is started securely (https). This file will contain your "Initial Admin Identity" and all your "Node Identity x" configured values:
authorizations.xml
The Authorizations.xml file is produced the first time and only the first time NiFi is started securely (https). NiFi will assign the access policies needed by your "Initial Admin Identity" and "Node Identity x" users/servers:
As you can see, your "Initial Admin Identity" user was granted the following resources/access policies:
| Resource: | NiFi UI Access Policy: | Details: |
| /flow (R) | view the UI | All users including admin must have this access policy in order to access and view the NiFi UI. |
| /restricted-components (W) | access restricted components | This access policy allows granted users the ability to add/configure NiFi components tagged as restricted on the canvas. |
| /tenants (R and W) | access users/user groups (view and modify) | This access policy allows granted users the ability to add/remove/modify new users and user groups to NiFi for authorization. |
| /policies (R and W) | access all policies (view and modify) | This access policy allows granted users the ability to add/remove various access policies for any users and user groups. |
| /controller (R and W) | access the controller (view and modify) | This access policy allows granted users the ability to view/modify the controller including Reporting Tasks, Controller Services, and Nodes in the Cluster |
You may notice a few additional access policies were granted to your admin user. This will only happen if the NiFi you have secured already had a an existing flow.xml.gz file. In this case the "Initial Admin Identity" is also granted access to view and modify the dataflow at the NiFi root canvas level. By default all sub NiFi process groups inherit their access policies from the parent process group. This effectively gives the admin user full access to the dataflow.
The "Node Identity x" servers are granted the following access policies:
| Resource: | NiFi UI Access Policy: | Details: |
| /proxy (R and W) | proxy user requests (view and modify) | Allows proxy machines to send requests on the behalf of others. All nodes in a NiFi cluster must be granted this access policy so users can make changes to the cluster while logged in to any of the NiFi Cluster's nodes. |
What do I do if i messed up my "Initial Admin Identity" or "Node Identity x" values when setting up my authorizers.xml file?
Its is common for users to incorrectly configure the value for the either the "Initial Admin Identity" or "Node Identity x" values.
Common mistakes include bad mapping patterns, case sensitivity issues (LDAP DNs always have the cn, ou, etc values in lowercase), white space issues between DN sections (cn=JohnDoe, ou=sme versus cn=JohnDoe,ou=sme).
You can use the nifi-user.log to identify the actual value being passed to the authorizer and then follow these steps:
- Correct your authorizers.xml configuration
- Delete or rename the current users.xml and authorizations.xml files on all of your NiFi nodes.
- restart all your nifi nodes
NiFi will generate new users.xml and authorizations.xml files from the corrected authorizers.xml file. You should only follow this procedure to correct issues when first setting up a secured NiFi. If an Admin was able to previously access your NiFi's canvas and add new users and granted access policies to those users, all those users and access policies will be lost if you delete the users.xml and authorizations.xml files.
Thanks,
Matt
