Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
Guru

Objective

With the release of NiFi 1.8.0, flowfiles that remain on a disconnected node can be rebalanced to other active nodes in the cluster via offloading.

Note: This tutorial assumes you have already setup a running NiFi cluster. The examples included are for a 2 node NiFi cluster.

Environment

This tutorial was tested using the following environment and components:

  • Mac OS X 10.11.6
  • Apache NiFi 1.8.0

Offload Nodes Tutorial

Queue FlowFiles

First, generate some queued flowfiles:

92996-1.png

In the Global Menu at the top left, select "Cluster" to see the Cluster Management dialog:

92997-2.png

The queued flowfiles are distributed between both nodes in the cluster:

92998-3.png

Disconnect Node

Select Disconnect for one of the nodes:

92999-4.png

The node is disconnected:

93000-5.png

Offload Node

Select Offload (93001-6.png) on the disconnected node. This will stop and terminate all processors and rebalance flowfiles to the other connected nodes in the cluster:

93002-7.png

Note: Offload would also stop transmitting on all remote process groups if they were in the flow.

When offloading is finished, all of the queued flowfiles are now on the active node:

93003-8.png

Delete & Decommission Node

An offloaded node can be either connected back to the cluster or deleted. Select Delete:

93004-9.png

Once deleted, the node cannot be rejoined to the cluster until it has been restarted. To decommission the node, stop/remove the NiFi service on the host.

1,017 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 06:04 AM
Updated by:
 
Contributors
Top Kudoed Authors