Skip to main content
ilyal
ilyal
Gold ⭐️
October 17, 2019
Solved

In NiFi queue with Load Balance Strategy RoundRobin flowfiles stuck

Hi All

If in connection(queue between two processors)  I switch on RoundRobin strategy flow files start to stuck there for minutes. 

Some investigation showed, that flow files stuck only on the primary node.

No any Warns or Errors in logs.

I am ready to provide any information

Please, any help.

Thanks

MattWho
MattWhoAnswer
Titanium ⭐️⭐️⭐️
October 17, 2019

@ilyal 

 

What version of NiFi are you running?  Is it Apache NiFi 1.8.x?

 

There are numerous bugs with the new load balanced connections feature.  The good number of these known bugs have been addressed between NiFi 1.9.0 and NiFi 1.9.2.

https://issues.apache.org/jira/browse/NIFI-5745

https://issues.apache.org/jira/browse/NIFI-5919

https://issues.apache.org/jira/browse/NIFI-5663

https://issues.apache.org/jira/browse/NIFI-5771

https://issues.apache.org/jira/browse/NIFI-6017

 

There are still some additional bugs that are fixed in NiFi 1.10.0

https://issues.apache.org/jira/browse/NIFI-6353

https://issues.apache.org/jira/browse/NIFI-6760

https://issues.apache.org/jira/browse/NIFI-6517

https://issues.apache.org/jira/browse/NIFI-6736

https://issues.apache.org/jira/browse/NIFI-6285

https://issues.apache.org/jira/browse/NIFI-6759

 

I strongly recommend upgrading to Apache NiFi 1.10 as a first step upgrading (releasing soon).

 

Hope this helps,
Matt

View in thread

13 replies

MattWho
Titanium ⭐️⭐️⭐️
MattWhoAnswer
Titanium ⭐️⭐️⭐️
October 17, 2019

@ilyal 

 

What version of NiFi are you running?  Is it Apache NiFi 1.8.x?

 

There are numerous bugs with the new load balanced connections feature.  The good number of these known bugs have been addressed between NiFi 1.9.0 and NiFi 1.9.2.

https://issues.apache.org/jira/browse/NIFI-5745

https://issues.apache.org/jira/browse/NIFI-5919

https://issues.apache.org/jira/browse/NIFI-5663

https://issues.apache.org/jira/browse/NIFI-5771

https://issues.apache.org/jira/browse/NIFI-6017

 

There are still some additional bugs that are fixed in NiFi 1.10.0

https://issues.apache.org/jira/browse/NIFI-6353

https://issues.apache.org/jira/browse/NIFI-6760

https://issues.apache.org/jira/browse/NIFI-6517

https://issues.apache.org/jira/browse/NIFI-6736

https://issues.apache.org/jira/browse/NIFI-6285

https://issues.apache.org/jira/browse/NIFI-6759

 

I strongly recommend upgrading to Apache NiFi 1.10 as a first step upgrading (releasing soon).

 

Hope this helps,
Matt

Translate
venu413
venu413
Silver ⭐️⭐️
January 6, 2020

Upgraded to 1.10 and now I see flow files are stuck with Load Balance strategy Partition By Attribute

Translate
MattWho
Titanium ⭐️⭐️⭐️
MattWho
Titanium ⭐️⭐️⭐️
January 7, 2020

@venu413 

If you open the NiFi summary UI (NiFi UI --> Global menu --> Summary), select the connections tab, locate this connection with the 54 queued flowfiles, and then click the cluster connection summary icon (Screen Shot 2020-01-07 at 4.47.14 PM.png)to far right, Are all 54 queued FlowFiles on same node?

Is anything being logged in the nifi-app.log on that node were these FlowFiles are queued?

Any observed errors in nifi-app.log during startup if you restart this node?

 

In your nifi.properties file, what values are configured for these properties:
nifi.cluster.load.balance.comms.timeout=
nifi.cluster.load.balance.connections.per.node=
nifi.cluster.load.balance.host=
nifi.cluster.load.balance.max.thread.count=
nifi.cluster.load.balance.port=
nifi.cluster.node.address=

If recommend that both the "nifi.cluster.node.address=" and "nifi.cluster.load.balance.host=" have been configured uniquely per node in your cluster to the resolvable hostname for the given node.  So if you node has a hostname of node1.mycompany.com, then this hostname should be used in both these properties in the NiFi running on that host.  Restart is needed anytime you edit the nifi.properties file.

Translate
Terms & ConditionsAccessibility statement