Objective This tutorial walks you through how to secure a NiFi instance using client certificates, configure access policies in NiFi, and then how to integrate it with a secured NiFi Registry to utilize versioned flows. A video version of this tutorial can be seen here: https://youtu.be/DSO12fhnZ90 Note: This tutorial assumes you have completed Setting Up a Secure Apache NiFi Registry which resulted in a NiFi Registry instance that has been secured with client certificates for two users: "sys_admin" and "test_user". Environment This tutorial was tested using the following environment and components: Mac OS X 10.11.6 Apache NiFi 1.5.0 Apache NiFi Registry 0.1.0 Apache NiFi Toolkit 1.5.0 Secure NiFi Configuration Download & Extract NiFi Tarball Download the tarball for the 1.5.0 Apache NiFi release: nifi-1.5.0-bin.tar.gz Extract the tar: tar xzvf nifi-1.5.0-bin.tar.gz NiFi Configuration From the article Setting Up a Secure Apache NiFi Registry, the TLS Toolkit was used to generate the following in the target directory: CN=sys_admin_OU=NIFI.p12 CN=sys_admin_OU=NIFI.p12.password CN=test_user_OU=NIFI.p12 CN=test_user_OU=NIFI.p12.password localhost nifi-cert.pem nifi-key.key and the following files in the localhost directory: keystore.jks nifi.properties truststore.jks Copy the keystore, nifi.properties and trustore to the conf directory of your NiFi install. Verify that in nifi.properties , the HTTP and HTTPS web properties are as follows: nifi.web.http.host= nifi.web.http.port= nifi.web.https.host=localhost nifi.web.https.port=9443 In the same NiFi conf directory, modify authorizers.xml in two places. In the userGroupProvider section, add the "sys_admin" DN to the "Initial Admin Identity 1" property: <property name="Initial User Identity 1">CN=sys_admin, OU=NIFI</property> Then in the accessPolicyProvider section, add the "sys_admin" DN to the "Initial Admin Identity" property: <property name="Initial Admin Identity">CN=sys_admin, OU=NIFI</property> Note: During this step, it is crucial that you specify the exact DN string used when the TLS Toolkit was invoked. A common error is entering "CN=sys_admin,OU=NIFI" which will not work as it has a missing space after the comma. Start NiFi and Open UI In a terminal window, navigate to the directory where NiFi was installed and run: ./bin/nifi.sh start Note: The "sys_admin" and "test_user" .p12 certificates should have already been added to the Keychain. Navigate to the NiFi UI in your web browser (Chrome used in the following examples): https://localhost:9443/nifi When prompted, select the "sys_admin" cert to add to your browser: You should now be able to view the NiFi UI as the "CN=sys_admin, OU=NIFI" user: Setup NiFi Access Policies for Sys_Admin The "sys_admin" user can view the NiFi UI, but has no permissions to make any changes. To correct this, select "Access Policies" (key icon) from the Operate Palette to modify the policies on the Root Process Group: For the "view the component" policy, select "Create" to create a new policy and add "sys_admin" to the policy: Do the same for the "modify the component" policy: Close the Access Policies window and the NiFi components toolbar is now enabled: Setup NiFi for Test_User The "test_user" also requires some setup to enable that user to access the NiFi UI. First go to the Global Menu in the top-right corner of the UI and select "Users": Add "CN=test_user, OU=NIFI" as a NiFi User: Close the "NiFi Users" dialog and select "Policies" from the Global menu: and add "test_user" to the "view the user interface" policy: Finally, similar to what was done previously for "sys_admin", select "Access Policies" from the Operate Palette and add "test_user" to the "view the component" policy: as well as to the "modify the component" policy: Navigate to the NiFi UI in another web browser (Safari used in the following examples): https://localhost:9443/nifi When prompted, select the "test_user" cert to add to your browser: You should now be able to view the NiFi UI as the "CN=test_user, OU=NIFI" user: Connect NiFi Registry Now we will connect our secure Registry to our secure NiFi instance. Return to the browser where you are logged into NiFi as the "sys_admin" user. Select "Controller Settings" from the top-right Global menu: Select the Registry Clients tab and the "+" button to add a new Registry Client. Enter a name and the URL of the Registry instance (https://localhost:18443): Localhost User Configuration in Registry The last configuration step is to add "CN=localhost, OU=NIFI" as a user in the Registry and configure its privileges. Log in to your secured Registry instance as "sys_admin". Note: As mentioned at the beginning of this tutorial, it is assumed that you have a secured NiFi Registry instance running already with "sys_admin" and "test_user" users from completing Setting Up a Secure Apache NiFi Registry. Select the Settings icon ( ) in the top right corner of the screen. Select "Users" at the top and then select the "Add User" button. Enter "CN=localhost, OU=NIFI" for the Identity/Username and select "Add": Select the pencil icon ( ) next to the "CN=localhost, OU=NIFI" user: In the side nav that appears, check the privileges "Read" buckets and "Can proxy user requests": Read to all buckets is necessary because NiFi itself checks the status of all the flows in NiFi to determine if they are in sync with the Registry.  For example, checking whether a new version of flow is available for upgrade. Proxy privileges are needed because when versioned flow actions occur, NiFi proxies the end user over to the Registry.  For example, when saving a version to the Registry it will be done as the actual user who performed this action in NiFi. You are now ready to start using NiFi and NiFi Registry together. NiFi and NiFi Registry Interaction Return to the browser where "sys_admin" is the NiFi user. "Sys_admin" has full registry bucket privileges. Create a new process group "PG_SA": Right click on the process group and select "Version-->Start version control": Since "sys_admin" has full privleges to buckets, the bucket "ABC" is is available to save the versioned flow to. Add the flow name "PG_SA_Flow" and select "Save": The process group is now under version control as evidenced by the green checkmark icon on the component: Now go to the browser where "test_user" is the NiFi user. "Test_user" only has read privileges to registry buckets. To demonstrate, create a process group "PG_TU": Right click on the process group and select "Version-->Start version control". However, in the "Save Flow Version" dialog, you will see that there are no available buckets to save the flow to: Close the dialog and drag a process group onto the canvas. Select "Import". In the Import Version dialog, the "ABC" bucket is accessible and the "PG_SA_Flow" in that bucket can be imported: Select Version 1 and then the "Import" button. A copy of "PG_SA_Flow" is added to the canvas: With the "sys_admin" having full access to the registry settings/administration, you can further explore the effects in NiFi after changing the privileges for "test_user". Additional Help If you would like to learn more about NiFi Registry functionality and working with versioned flows in NiFi, see the following articles: Versioned DataFlows with Apache NiFi 1.5 and Apache NiFi Registry 0.1.0 Apache NiFi - How do I deploy my flow? Or documentation: Apache NiFi Registry User Guide Apache NiFi Registry System Administrator's Guide Versioning a DataFlow (Apache NiFi User Guide) ... View more

Objective This tutorial walks you through how to install and secure a NiFi Registry using client certificates. A quick example of modifying user privileges in the Registry is also included. A video version of this tutorial can be seen here: https://youtu.be/qD03ao3R-a4 Note: To learn the basics of setting up an unsecured Registry and integrating with Apache NiFi see the HCC article Versioned DataFlows with Apache NiFi 1.5 and Apache NiFi Registry 0.1.0. Environment This tutorial was tested using the following environment and components: Mac OS X 10.11.6 Apache NiFi Registry 0.1.0 Apache NiFi Toolkit 1.5.0 Secure NiFi Registry Configuration Download & Extract Tarballs Download the tarball for the 0.1.0 Registry release: nifi-registry-0.1.0-bin.tar.gz and the tarball for the 1.5.0 NiFi Toolkit: nifi-toolkit-1.5.0-bin.tar.gz Extract the tars: tar xzvf nifi-registry-0.1.0-bin.tar.gz tar xzvf nifi-toolkit-1.5.0-bin.tar.gz Generate Configuration and Certificate Files We will use the Apache NiFi TLS Toolkit to generate the necessary keystore, truststore, and client certificates. In this tutorial, we will create certs for two users: "sys_admin" and "test_user". The user “sys_admin” will have full access to the registry while “test_user” will be configured to have targeted access in the registry. In the directory of your NiFi Toolkit install, run the following command: ./bin/tls-toolkit.sh standalone -n "localhost" -C "CN=sys_admin, OU=NIFI" -o target Note: To see the usage information for the TLS Toolkit, run: ./bin/tls-toolkit.sh standalone -h . TLS Toolkit generates the following in the target directory: CN=sys_admin_OU=NIFI.p12 CN=sys_admin_OU=NIFI.p12.password localhost nifi-cert.pem nifi-key.key The localhost directory contains: keystore.jks nifi.properties truststore.jks Registry Configuration Copy the keystore and trustore to the conf directory of your Registry install. Copy the values of the keystore and truststore properties from the nifi.properties file: nifi.security.keystore=./conf/keystore.jks nifi.security.keystoreType=jks nifi.security.keystorePasswd=taceJshGdkyBRy4B7mwaSnM3AkbN7ffewjn3nVIGidw nifi.security.keyPasswd=taceJshGdkyBRy4B7mwaSnM3AkbN7ffewjn3nVIGidw nifi.security.truststore=./conf/truststore.jks nifi.security.truststoreType=jks nifi.security.truststorePasswd=WJwg6F2jmUcvpxRHDiseNRc/VV59WOS+SdrZ5amtnsE into the values for the equivalent properties in the nifi-registry.properties file: nifi.registry.security.keystore=./conf/keystore.jks nifi.registry.security.keystoreType=jks nifi.registry.security.keystorePasswd=taceJshGdkyBRy4B7mwaSnM3AkbN7ffewjn3nVIGidw nifi.registry.security.keyPasswd=taceJshGdkyBRy4B7mwaSnM3AkbN7ffewjn3nVIGidw nifi.registry.security.truststore=./conf/truststore.jks nifi.registry.security.truststoreType=jks nifi.registry.security.truststorePasswd=WJwg6F2jmUcvpxRHDiseNRc/VV59WOS+SdrZ5amtnsE While you are in nifi-registry.properties , modify the HTTP and HTTPS web properties as follows: nifi.registry.web.http.host= nifi.registry.web.http.port= nifi.registry.web.https.host=localhost nifi.registry.web.https.port=18443 In the same Registry conf directory, modify authorizers.xml in two places. First in the userGroupProvider section, add the "sys_admin" DN to the "Initial Admin Identity 1" property: <property name="Initial User Identity 1">CN=sys_admin, OU=NIFI</property> Then in the accessPolicyProvider section, add the "sys_admin" DN to the "Initial Admin Identity" property: <property name="Initial Admin Identity">CN=sys_admin, OU=NIFI</property> Note: During this step, it is crucial that you specify the exact DN string used when the TLS Toolkit was invoked. A common error is entering "CN=sys_admin,OU=NIFI" which will not work as it has a missing space. Add Certificate to Keychain Double-click on the .p12 file that was generated by the TLS Toolkit. When prompted, provide the password from the .password file. Start the Registry In a terminal window, navigate to the directory where NiFi Registry was installed and run: ./bin/nifi-registry.sh start Open Registry UI Navigate to the registry UI in your web browser (Chrome used in the following examples): https://localhost:18443/nifi-registry When prompted, select the "sys_admin" cert to add to your browser: When prompted, enter your "login" keychain password: You should now be able to view the Registry UI as the "CN=sys_admin, OU=NIFI" user: Registry Administration The "sys_admin" user has full access to the registry. Here are some examples of administration functions immediately available. Bucket Creation Select the Settings icon ( ) in the top right corner of the screen. In the Buckets window that appears, select the "New Bucket" button. In the dialog that appears, enter the bucket name "ABC" and select the "Create" button. The "ABC" bucket is created: User Administration Select "Users" at the top of the UI to access the user administration area of the Registry: Select the pencil icon ( ) next to the "CN=sys_admin, OU=NIFI" user. This will open a side nav that shows the Special Privileges and group Membership: You can see that the "sys_admin" was given all special privileges as the Initial Admin Identity (IAI). The privileges for the IAI are not editable. Let's create a second user to see how bucket access can be restricted by modifying these privileges. Second User Creation Close the side nav and select the "Add User" button. Enter "CN=test_user, OU=NIFI" in the Identity field and select the "Add" button: "CN=test_user", OU=NIFI" user is created: Second User Certificate Next we need a client certificate for "test_user". Return to the directory of your NiFi Toolkit installation and run: ./bin/tls-toolkit.sh standalone -C "CN=test_user, OU=NIFI" -o target NOTE:The output directory must be set to target in order for the existing CA certificate in that directory to be used. TLS Toolkit generates the following additional files in the target directory: CN=test_user_OU=NIFI.p12 CN=test_user_OU=NIFI.p12.password Add the .p12 cert to the Keychain as described earlier. However, choose a different browser this time to access the UI (Safari in the following examples): https://localhost:18443/nifi-registry Add the client certificate to the browser: You should now be able to view the Registry UI as the "CN=test_user, OU=NIFI" user: You can see that "test_user" has no access to Settings. Return to the Chrome browser where "sys_admin" is the user. Give "test_user" read-only bucket privileges: Return to the Safari browser where "test_user" is the user. Reload the browser. Select the Settings icon which is now available. The ABC bucket is now visible, but note that the Action to delete the bucket is not enabled, which is consistent with the privileges given to this user: Additional Help If you would like to learn more about NiFi Registry functionality and working with versioned flows in NiFi, see the following articles: Versioned DataFlows with Apache NiFi 1.5 and Apache NiFi Registry 0.1.0 Apache NiFi - How do I deploy my flow? Or documentation: Apache NiFi Registry User Guide Apache NiFi Registry System Administrator's Guide Versioning a DataFlow (Apache NiFi User Guide) ... View more

Objective This article highlights some of the latest UI enhancements added in Apache NiFi 1.5.0. Environment The examples shown in the article utilized the following environment and components: Mac OS X 10.11.6 Apache NiFi 1.5.0 "Primary Node" Processors Identification In a NiFi Cluster, processors that have been configured for "Primary node" execution are now identified in the UI by a "P". On the canvas, the "P" is visible next to the processor icon: The "P" is also shown in the Processors tab on the Summary page, specifically in the Name column: Finding Processors Quickly in the Summary Page If your flow has hundreds of processors, it can be difficult differentiating between them in the Summary page (accessible from the top-right Global menu). On the Processors tab, a "Process Group" column has been added to display the name of the parent process group containing the component: Additionally, when hovering over the "Go to location" button the tooltip now includes the path of the component. NiFi Registry Integration NiFi 1.5.0 is the first release to integrate with the Apache NiFi Registry. NiFi dataflows can now be versioned on the process group level and easily deployed across different NiFi instances. More information can be found in the HCC article "Versioned DataFlows with Apache NiFi 1.5 and Apache NiFi Registry 0.1.0" and in the "Versioning a Dataflow" section of the NiFi User Guide. However, here are some related UI changes to highlight. Connecting a Registry Client The NiFi Settings window (accessible from Controller Settings in the top-right Global menu) now has a "Registry Clients" tab where you can connect NiFi to a NiFi Registry: Importing a Flow If your NiFi instance is connected to an active Registry, when adding a process group to the canvas there is also an option to "Import" a versioned flow: Selecting "Import" prompts the user to choose a version of a flow to add to the canvas: Version States There are new icons that show: the version state of an individual process group the count of the statuses of versioned process groups within a process group the count of the statuses of versioned process groups in the root process group Here are the meanings of each icon/state: Up to date Locally modified Stale Locally modified and stale Sync failure Version state information is also shown in the "Process Groups" tab of the Summary Page: As mentioned previously, more information regarding NiFi and NiFi Registry integration can be found in the "Versioning a Dataflow" section of the NiFi User Guide. ... View more