The InvokeHttp Processor's 'Retry' option is causing the flow to queue up after the retry attempts.

[#1]

I have configured an InvokeHttp Processor with the 'Retry' relationship enabled to capture HTTP status codes between 500 and 599 and initiate retries. I connected the 'Retry' relationship back to the InvokeHTTP processor to enable automatic reattempts. The number of retry attempts is set to 3. However, after all retry attempts are exhausted, the process does not complete and ends up queuing in the Retry relationship, as shown below:

Could you please help how to fix it?

[#2]

@s198 Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi experts @SAMSAL @MattWho  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

[#3]

This may be related to https://issues.apache.org/jira/browse/NIFI-11783 which was fixed in CFM 2.1.6. Is upgrading your CFM an option?

[#4]

@s198 

If I am following your description correctly, It appears to be working as designed.

 The number of retry attempts is set to 3. However, after all retry attempts are exhausted, the process does not complete and ends up queuing in the Retry relationship.

When you set "Retry" on a relationship, it tell NiFi to attempt that many retries before on the component BEFORE routing the FlowFile to the relationship (So FlowFile stays on current inbound connection and you see the hour glass indicating a FlowFile is penalized on that connection).  See in your dataflow InvokeHTTP will retry up to 3 times to process the inbound FlowFile. If processing of the FlowFile results in a "retry" relationship destination, it will get penalized again until configured number of retries completes and the route to "retry" relationship connection.  And that is waht we see happening in your dataflow.  After those three attempts the FlowFile get transferred to "retry" relationship which you have in a connection looping back on the processor.  We then see the hour glass there meaning the 3 reties is happening again.  This will continue forever unless that FlowFile is eventually successful.

I think what you really wanted to do is retry 3 times and if all three times result in routing to retry relationship, terminate the FlowFile (delete it).    So in this case you would check both "terminate" and "retry" boxed on the retry relationship.  After 3 unsuccess FlowFile get terminated.

If you want to retain the FlowFile after 3 failed retry attempts, don't loop the "retry" relationship back on the processor, but instead route to some other processor for your error handling/notification/etc...

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

[#5]

Thank you @MattWho  for your valuable input.

Yes, as you mentioned, my requirement is to retry 3 times, and if all three attempts result in routing to the retry relationship, terminate the FlowFile.

I tried both options you mentioned.

The first option doesn’t seem to be possible. I tried checking both the "terminate" and "retry" boxes on the retry relationship, but an error popped up: "Cannot automatically terminate 'Retry' relationship because a connection already exists with this relationship." Since I’ve already connected the retry relationship back to the InvokeHttp processor, it’s not allowing me to enable "terminate."

Could you please provide more input on this option?

The second option is working fine. Instead of looping the "retry" relationship back to the processor, I routed it to another processor. After the 3 retry attempts, it routes to the next processor and terminates the FlowFile.