Apache NiFi Sharing of load in a cluster not happening

varungupta — Wed, 14 Aug 2024 08:38:38 GMT

In Apache NiFi, We are interacting to SOAP api through powershell. As API expect attachment with MTOM.
Now if we load 10000 request we can see only one server is acting on those requeset while other remainnig 2 server are sitting idle. Is there way to share the load acrooss all three servers?

Re: Apache NiFi Sharing of load in a cluster not happening

florence0239 — Fri, 16 Aug 2024 08:44:07 GMT

@varungupta wrote:
In Apache NiFi, We are interacting to SOAP api through powershell. As API expect attachment with MTOM. Official Site
Now if we load 10000 request we can see only one server is acting on those requeset while other remainnig 2 server are sitting idle. Is there way to share the load acrooss all three servers?

Hello,

It sounds like you’re encountering a load balancing issue in your Apache NiFi cluster. To ensure that all three servers share the load, you can use NiFi’s built-in load balancing features. Here are some steps to help you achieve this:

Enable Load-Balanced Connections: NiFi allows you to configure connections to distribute data across the cluster. You can set the load balancing strategy for each connection. The available strategies are:
Do Not Load Balance: Default setting, no load balancing.
Round Robin: Distributes FlowFiles evenly across all nodes.
Single Node: Sends all FlowFiles to a single node.
Partition by Attribute: Distributes FlowFiles based on a specific attribute.
Configure Load Balancing:
Open your NiFi flow and select the connection you want to load balance.
In the connection settings, go to the Load Balancing tab.
Choose the appropriate load balancing strategy, such as Round Robin.
Check Node Status: Ensure all nodes in your cluster are active and properly connected. You can check the status of your nodes in the NiFi Cluster Manager.
Monitor Performance: Use NiFi’s monitoring tools to observe the distribution of FlowFiles and ensure that the load is balanced across all nodes.
Here’s a quick example of how to set up a load-balanced connection:

Select the Connection: Click on the connection between processors in your NiFi flow.
Load Balancing Tab: In the connection settings, navigate to the Load Balancing tab.
Choose Strategy: Select Round Robin to distribute FlowFiles evenly across all nodes.
By following these steps, you should be able to distribute the load more evenly across your NiFi cluster, ensuring that all servers are utilized efficiently

Hope this will help you.
Bets regards,
florence0239

question Apache NiFi Sharing of load in a cluster not happening in Support Questions

Apache NiFi Sharing of load in a cluster not happening

Re: Apache NiFi Sharing of load in a cluster not happening