Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
Labels (1)
avatar
Guru

Objective

This tutorial walks you through how to secure a NiFi instance using client certificates, configure access policies in NiFi, and then how to integrate it with a secured NiFi Registry to utilize versioned flows.

A video version of this tutorial can be seen here: https://youtu.be/DSO12fhnZ90

Note: This tutorial assumes you have completed Setting Up a Secure Apache NiFi Registry which resulted in a NiFi Registry instance that has been secured with client certificates for two users: "sys_admin" and "test_user".

Environment

This tutorial was tested using the following environment and components:

  • Mac OS X 10.11.6
  • Apache NiFi 1.5.0
  • Apache NiFi Registry 0.1.0
  • Apache NiFi Toolkit 1.5.0

Secure NiFi Configuration

Download & Extract NiFi Tarball

Download the tarball for the 1.5.0 Apache NiFi release:

nifi-1.5.0-bin.tar.gz

Extract the tar:

 tar xzvf nifi-1.5.0-bin.tar.gz

NiFi Configuration

From the article Setting Up a Secure Apache NiFi Registry, the TLS Toolkit was used to generate the following in the target directory:

CN=sys_admin_OU=NIFI.p12
CN=sys_admin_OU=NIFI.p12.password
CN=test_user_OU=NIFI.p12
CN=test_user_OU=NIFI.p12.password
localhost
nifi-cert.pem
nifi-key.key

and the following files in the localhost directory:

keystore.jks
nifi.properties
truststore.jks

Copy the keystore, nifi.properties and trustore to the conf directory of your NiFi install.

Verify that in nifi.properties, the HTTP and HTTPS web properties are as follows:

 nifi.web.http.host=
 nifi.web.http.port=
 nifi.web.https.host=localhost
 nifi.web.https.port=9443

In the same NiFi conf directory, modify authorizers.xml in two places. In the userGroupProvider section, add the "sys_admin" DN to the "Initial Admin Identity 1" property:

 <property name="Initial User Identity 1">CN=sys_admin, OU=NIFI</property>

Then in the accessPolicyProvider section, add the "sys_admin" DN to the "Initial Admin Identity" property:

 <property name="Initial Admin Identity">CN=sys_admin, OU=NIFI</property>

Note: During this step, it is crucial that you specify the exact DN string used when the TLS Toolkit was invoked. A common error is entering "CN=sys_admin,OU=NIFI" which will not work as it has a missing space after the comma.

Start NiFi and Open UI

In a terminal window, navigate to the directory where NiFi was installed and run:

 ./bin/nifi.sh start

Note: The "sys_admin" and "test_user" .p12 certificates should have already been added to the Keychain.

Navigate to the NiFi UI in your web browser (Chrome used in the following examples):

https://localhost:9443/nifi

When prompted, select the "sys_admin" cert to add to your browser:

56709-1-add-cert-to-browser.png

You should now be able to view the NiFi UI as the "CN=sys_admin, OU=NIFI" user:

56710-2-blank-canvas-nifi.png

Setup NiFi Access Policies for Sys_Admin

The "sys_admin" user can view the NiFi UI, but has no permissions to make any changes. To correct this, select "Access Policies" (key icon) from the Operate Palette to modify the policies on the Root Process Group:

56711-3-root-pg-policies.png

For the "view the component" policy, select "Create" to create a new policy and add "sys_admin" to the policy:

56712-4-root-pg-view-sys-admin.png

Do the same for the "modify the component" policy:

56713-5-root-pg-modify-sys-admin.png

Close the Access Policies window and the NiFi components toolbar is now enabled:

56714-6-ui-enabled.png

Setup NiFi for Test_User

The "test_user" also requires some setup to enable that user to access the NiFi UI.

First go to the Global Menu in the top-right corner of the UI and select "Users":

56715-7-global-menu-users.png

Add "CN=test_user, OU=NIFI" as a NiFi User:

56716-8-add-test-user.png

Close the "NiFi Users" dialog and select "Policies" from the Global menu:

56717-9-global-menu-policies.png

and add "test_user" to the "view the user interface" policy:

56718-10-view-ui-test-user.png

Finally, similar to what was done previously for "sys_admin", select "Access Policies" from the Operate Palette and add "test_user" to the "view the component" policy:

56719-11-root-pg-view-test-user.png

as well as to the "modify the component" policy:

56720-12-root-pg-modify-test-user.png

Navigate to the NiFi UI in another web browser (Safari used in the following examples):

https://localhost:9443/nifi

When prompted, select the "test_user" cert to add to your browser:

56721-13-add-test-user-cert.png

You should now be able to view the NiFi UI as the "CN=test_user, OU=NIFI" user:

56722-14-nifi-ui-test-user.png

Connect NiFi Registry

Now we will connect our secure Registry to our secure NiFi instance.

Return to the browser where you are logged into NiFi as the "sys_admin" user. Select "Controller Settings" from the top-right Global menu:

56723-15-global-menu-controller-settings.png

Select the Registry Clients tab and the "+" button to add a new Registry Client. Enter a name and the URL of the Registry instance (https://localhost:18443):

56724-16-add-local-secured-registry-client.png

Localhost User Configuration in Registry

The last configuration step is to add "CN=localhost, OU=NIFI" as a user in the Registry and configure its privileges.

Log in to your secured Registry instance as "sys_admin".

Note: As mentioned at the beginning of this tutorial, it is assumed that you have a secured NiFi Registry instance running already with "sys_admin" and "test_user" users from completing Setting Up a Secure Apache NiFi Registry.

Select the Settings icon ( 56725-iconsettings.png) in the top right corner of the screen. Select "Users" at the top and then select the "Add User" button.

56726-17-add-localhost-user.png

Enter "CN=localhost, OU=NIFI" for the Identity/Username and select "Add":

56727-18-localhost-user-identity.png

Select the pencil icon ( 56728-iconmanage.png) next to the "CN=localhost, OU=NIFI" user:

56729-19-manage-localhost-user.png

In the side nav that appears, check the privileges "Read" buckets and "Can proxy user requests":

56730-20-localhost-user-special-privileges.png

Read to all buckets is necessary because NiFi itself checks the status of all the flows in NiFi to determine if they are in sync with the Registry.  For example, checking whether a new version of flow is available for upgrade. Proxy privileges are needed because when versioned flow actions occur, NiFi proxies the end user over to the Registry.  For example, when saving a version to the Registry it will be done as the actual user who performed this action in NiFi.

You are now ready to start using NiFi and NiFi Registry together.

NiFi and NiFi Registry Interaction

Return to the browser where "sys_admin" is the NiFi user. "Sys_admin" has full registry bucket privileges. Create a new process group "PG_SA":

56731-21-sys-admin-user-create-pg.png

Right click on the process group and select "Version-->Start version control":

56732-22-pg-sa-version-control.png

Since "sys_admin" has full privleges to buckets, the bucket "ABC" is is available to save the versioned flow to. Add the flow name "PG_SA_Flow" and select "Save":

56733-23-pg-sa-save-flow.png

The process group is now under version control as evidenced by the green checkmark icon on the component:

56734-24-pg-sa-green-checkmark.png

Now go to the browser where "test_user" is the NiFi user. "Test_user" only has read privileges to registry buckets. To demonstrate, create a process group "PG_TU":

56735-25-test-user-create-pg.png

Right click on the process group and select "Version-->Start version control". However, in the "Save Flow Version" dialog, you will see that there are no available buckets to save the flow to:

56736-26-pg-tu-save-flow-no-buckets.png

Close the dialog and drag a process group onto the canvas. Select "Import". In the Import Version dialog, the "ABC" bucket is accessible and the "PG_SA_Flow" in that bucket can be imported:

56737-27-pg-sa-flow-import-dialog.png

Select Version 1 and then the "Import" button. A copy of "PG_SA_Flow" is added to the canvas:

56738-28-pg-sa-flow-added.png

With the "sys_admin" having full access to the registry settings/administration, you can further explore the effects in NiFi after changing the privileges for "test_user".

Additional Help

If you would like to learn more about NiFi Registry functionality and working with versioned flows in NiFi, see the following articles:

Or documentation:

19,547 Views
Comments
avatar
Guru

If the Localhost user is not configured properly in the Registry, here is an error you will see in NiFi when trying to start version control on a process group:

92546-untrusted-proxy-error.png

avatar
Explorer

Very useful the blog post !
How is possible to connect multiple nifi environments (like dev, testing, prod) to a single (secured nifi-registry)?
Based on this example you copied  keystore, nifi.properties and trustore from nifi-registry to nify. 

How can I do if my different nifi instances are already configured and have different keystore / truststore ?

Thanks !

avatar
Super Guru

@Pe If you have a new question, you can post it and reference the article.  However, since im already responding, I will be brief.  Yes, all dev, test,prod to usually Prod NiFi Registry.  Or sometimes all to Dev, your preference.  To connect to existing keystores, would just add the new certs from the nifi registry host to all the other existing keystores/truststores.

avatar
New Contributor

Great article!

Is Registry also keeping track of the user that committed the changes?

If yes, does this also work in the case that NiFi is configured to use OIDC instead of client certificates? (i.e. different usernames in NiFi than in Registry)

avatar
New Contributor
Hi Andrew, thanks for the tutorial. It was very useful. It's strange that the official nifi-registry documentation doesn't cover any of this. Anyway, I feel I should add a missing step to the instructions.
 
Chances are, the "CN=localhost,OU=nifi" is not going to be valid in real-world scenarios. To figure out what exactly the user name should be, you can check the nifi-app.log where you will see a message like this:
 
org.apache.nifi.registry.client.NiFiRegistryException: Error retrieving flow snapshot: Unknown user with identity 'CN=host1, OU=ACME, O=ACME, L=Canberra, ST=Australian Capital Territory, C=AU'. Contact the system administrator.
 
Simply use that identity instead of "CN=localhost,OU=nifi".
avatar
Rising Star

Hi,

 

I am trying to implemented this blog and so far have successfully completed till version controlling. However, I get an error when I am trying to import the flow in the PG. The prompted error says that FlowID does not exists in Bucket ID whereas I can see that the specific flow is present in the nifi registry in the UI. Can anyone help explain why this is happening?

 

Additional info:

Nifi version: 1.20.0

Nifi registry: 1.22.0 (securely connected to Nifi and buckets are given access to the CNs)

Log: Error retrieving flow snapshot: An unexpected error has occurred. Please check the logs for additional details.
at org.apache.nifi.registry.client.impl.AbstractJerseyClient.executeAction(AbstractJerseyClient.java:117) Caused by: javax.ws.rs.InternalServerErrorException: HTTP 500 Internal Server Error

o.a.n.r.web.mapper.ThrowableMapper An unexpected error has occurred: java.lang.NullPointerException. Returning Internal Server Error response.

Version history
Last update:
‎08-17-2019 09:00 AM
Updated by:
Contributors