Skip to main content
ilyal
ilyal
Gold ⭐️
October 17, 2019
Solved

In NiFi queue with Load Balance Strategy RoundRobin flowfiles stuck

Hi All

If in connection(queue between two processors)  I switch on RoundRobin strategy flow files start to stuck there for minutes. 

Some investigation showed, that flow files stuck only on the primary node.

No any Warns or Errors in logs.

I am ready to provide any information

Please, any help.

Thanks

MattWho
MattWhoAnswer
Titanium ⭐️⭐️⭐️
October 17, 2019

@ilyal 

 

What version of NiFi are you running?  Is it Apache NiFi 1.8.x?

 

There are numerous bugs with the new load balanced connections feature.  The good number of these known bugs have been addressed between NiFi 1.9.0 and NiFi 1.9.2.

https://issues.apache.org/jira/browse/NIFI-5745

https://issues.apache.org/jira/browse/NIFI-5919

https://issues.apache.org/jira/browse/NIFI-5663

https://issues.apache.org/jira/browse/NIFI-5771

https://issues.apache.org/jira/browse/NIFI-6017

 

There are still some additional bugs that are fixed in NiFi 1.10.0

https://issues.apache.org/jira/browse/NIFI-6353

https://issues.apache.org/jira/browse/NIFI-6760

https://issues.apache.org/jira/browse/NIFI-6517

https://issues.apache.org/jira/browse/NIFI-6736

https://issues.apache.org/jira/browse/NIFI-6285

https://issues.apache.org/jira/browse/NIFI-6759

 

I strongly recommend upgrading to Apache NiFi 1.10 as a first step upgrading (releasing soon).

 

Hope this helps,
Matt

View in thread

13 replies

MattWho
Titanium ⭐️⭐️⭐️
MattWhoAnswer
Titanium ⭐️⭐️⭐️
October 17, 2019

@ilyal 

 

What version of NiFi are you running?  Is it Apache NiFi 1.8.x?

 

There are numerous bugs with the new load balanced connections feature.  The good number of these known bugs have been addressed between NiFi 1.9.0 and NiFi 1.9.2.

https://issues.apache.org/jira/browse/NIFI-5745

https://issues.apache.org/jira/browse/NIFI-5919

https://issues.apache.org/jira/browse/NIFI-5663

https://issues.apache.org/jira/browse/NIFI-5771

https://issues.apache.org/jira/browse/NIFI-6017

 

There are still some additional bugs that are fixed in NiFi 1.10.0

https://issues.apache.org/jira/browse/NIFI-6353

https://issues.apache.org/jira/browse/NIFI-6760

https://issues.apache.org/jira/browse/NIFI-6517

https://issues.apache.org/jira/browse/NIFI-6736

https://issues.apache.org/jira/browse/NIFI-6285

https://issues.apache.org/jira/browse/NIFI-6759

 

I strongly recommend upgrading to Apache NiFi 1.10 as a first step upgrading (releasing soon).

 

Hope this helps,
Matt

Translate
venu413
venu413
Silver ⭐️⭐️
January 6, 2020

Upgraded to 1.10 and now I see flow files are stuck with Load Balance strategy Partition By Attribute

Translate
JatinSab
JatinSab
Silver ⭐️⭐️
February 10, 2020

@MattWho Here are the values of properties asked

nifi.cluster.load.balance.host=nifi-dev-0.nifi-dev
nifi.cluster.load.balance.port=6342
nifi.cluster.load.balance.connections.per.node=10
nifi.cluster.load.balance.max.thread.count=8
nifi.cluster.load.balance.comms.timeout=300 sec

nifi.cluster.node.address=nifi-dev-0.nifi-dev

 

Still the issue is appearing. 
Actually we have a Kubernetes Cluster using NiFi 1.11.1 image. However, it still appears to happen

Translate
Terms & ConditionsAccessibility statement