Offload NiFi Cluster Nodes using the UI (NiFi 1.8.0)

[#1]

Objective

With the release of NiFi 1.8.0, flowfiles that remain on a disconnected node can be rebalanced to other active nodes in the cluster via offloading.

Note: This tutorial assumes you have already setup a running NiFi cluster. The examples included are for a 2 node NiFi cluster.

Environment

This tutorial was tested using the following environment and components:

• Mac OS X 10.11.6
• Apache NiFi 1.8.0

Offload Nodes Tutorial

Queue FlowFiles

First, generate some queued flowfiles:

In the Global Menu at the top left, select "Cluster" to see the Cluster Management dialog:

The queued flowfiles are distributed between both nodes in the cluster:

Disconnect Node

Select Disconnect for one of the nodes:

The node is disconnected:

Offload Node

Select Offload () on the disconnected node. This will stop and terminate all processors and rebalance flowfiles to the other connected nodes in the cluster:

Note: Offload would also stop transmitting on all remote process groups if they were in the flow.

When offloading is finished, all of the queued flowfiles are now on the active node:

Delete & Decommission Node

An offloaded node can be either connected back to the cluster or deleted. Select Delete:

Once deleted, the node cannot be rejoined to the cluster until it has been restarted. To decommission the node, stop/remove the NiFi service on the host.

[#2]

The tutorial was really informative. Can you please help me resolve the following doubts(Im a beginner in nifi).

1) Can we offload the flowfiles if the nifi process on that node is killed(or stopped) ? When I tried, the Status remains in OFFLOADING state instead of changing to OFFLOADED state.

2) Can we initiate the offloading process through curl script? as we do in the case of processors, to start and stop them?

3) Another doubt apart from the tutorial is, can we change the state of a processor if one node is disconnected from the cluster? When tried, it displays the following message :

"Cluster is unable to service request to change flow: Node is currently disconnected"

Please do mention if we can achieve this by any means.Thank you..

[#3]

Can we offload the flowfiles if the nifi process on that node is killed(or stopped) ? When I tried, the Status remains in OFFLOADING state instead of changing to OFFLOADED state.