Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
Guru

Objective

This tutorial walks you through how to install and setup a local Apache NiFi Registry to integrate with Apache NiFi and start using versioned NiFi dataflows. It assumes basic experience with NiFi but little to no experience with NiFi Registry.

A video version of this tutorial can be seen here: https://youtu.be/X_qhRVChjZY

Environment

This tutorial was tested using the following environment and components:

  • Mac OS X 10.11.6
  • Apache NiFi 1.5.0
  • Apache NiFi Registry 0.1.0

Note: Apache NiFi 1.5.0 is the first NiFi release to support integration with the NiFi Registry. Nifi Registry 0.1.0 is the first and currently only version of the application.

Apache NiFi Registry Configuration

Registry Installation

Download the tarball of the 0.1.0 Registry release:

nifi-registry-0.1.0-bin.tar.gz

Extract the tar:

tar xzvf nifi-registry-0.1.0-bin.tar.gz

Start the Registry

In a terminal window, navigate to the directory where NiFi Registry was installed. Run:

bin/nifi-registry.sh start

Open Registry UI

Navigate to the registry UI in your browser:

http://localhost:18080/nifi-registry

Note:By default the registry is unsecured. The port can be changed by editing the nifi-registry.properties file in the NiFi Registry conf directory (the exact property to change is nifi.registry.web.http.port), but the default port is 18080.

Bucket Creation

A Bucket is a container that stores and organizes flows in the Registry. The Registry is empty as there are no buckets/flows yet.

56386-1-empty-registry.png

To create a bucket, select the Settings icon (56387-2-iconsettings.png) in the top right corner of the screen. In the Buckets window that appears, select the "New Bucket" button.

56388-3-new-test-bucket.png

Enter the bucket name "Test" and select the "Create" button.

56389-4-test-bucket-dialog.png

The "Test" bucket is created:

56390-5-test-bucket.png

There are no permissions configured by default, so anyone is able to view, create and modify buckets in this instance. For information on securing the Registry, see the NiFi Registry System Administrator’s Guide.

Apache NiFi Configuration

Connect NiFi to the Registry

With the Registry is running, we can tell NiFi about it.

In NiFi, select "Controller Settings" from the top-right Global menu:

56391-6-controller-settings-selection.png

Select the Registry Clients tab and the "+" button to add a new Registry Client. Enter a name and the URL of the Registry instance (http://localhost:18080):

56399-7-local-registry.png

Versioned DataFlows

Start Version Control on a Process Group

NiFi can now place a process group under version control which saves it as a flow resource in the Registry.

Right-click on a process group and select "Version→Start version control" from the context menu:

56393-8-abcd-process-group-menu.png

The local registry instance and "Test" bucket are chosen by default to store your flow since they are the only registry connected and bucket available. Enter a flow name, flow description, comments and select "Save":

56394-9-save-abcd-flow-dialog.png

As indicated by the Version State icon (56395-10-iconuptodate.png) in the top left corner of the component, the process group is now saved as a versioned flow in the registry.

56396-11-abcd-flow-saved.png

Go back to the Registry UI and return to the main page to see the versioned flow you just saved (a refresh may be required):

56397-12-abcd-flow-in-test-bucket.png

Save Changes to a Versioned Flow

Changes made to the versioned process group can be reviewed, reverted or saved.

For example, if changes are made to the ABCD flow, the Version State changes to "Locally modified" (56400-13-iconlocallymodified.png). The right-click menu will now show the options "Commit local changes", "Show local changes" or "Revert local changes":

56401-14-changed-flow-options.png

Select "Show local changes" to see the details of the changes made:

56402-15-abcd-flow-changes.png

Return to the context menu and select "Commit local changes". Enter comments and select "Save" to save the changes:

56403-16-abcd-save-flow-version-2.png

Version 2 of the flow is saved:

56404-17-abcd-version-2.png

Note: Some actions made to the versioned process group are not considered local changes. More information can be found in the Managing Local Changes section of the NiFi User Guide.

Import a Versioned Flow

With a flow existing in the Registry, we can use it to illustrate how to import a versioned process group.

In NiFi, select Process Group from the Components toolbar and drag it onto the canvas:

56405-18-drag-process-group.png

Instead of entering a name, click the Import link:

56406-19-import-flow-from-registry.png

Choose the version of the flow you want imported and select "Import":

56407-20-import-abcd-version-2.png

A second identical PG is now added:

56408-21-two-abcd-flows.png

Help

To learn more about NiFi Registry functionality and working with versioned flows in NiFi, see the following links:

1,828 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 09:13 AM
Updated by:
 
Contributors
Top Kudoed Authors