Member since
11-22-2017
1
Post
0
Kudos Received
0
Solutions
11-22-2017
03:42 PM
In my experience, the retry loop pattern has led to a deadlock when
the number of failed flow files approaches the size of the queues
entering and exiting the loop. The issue is that both of those queues
fill up, and NiFi is not able to pick up on the fact that it can cycle
flowfiles between the queues. This also has the unfortunate side effect
of blocking any flowfiles from entering the initial error-prone
processor, as they could potentially fail, and enter a full queue. The
best solution I have as of now is to simply duplicate any failure prone
processors, considering I usually only want to retry 1-3 times. See: http://apache-nifi-users-list.2361937.n4.nabble.com/Back-pressure-deadlock-td3274.html
... View more