question Re: NiFi: invokehttp retries in Archives of Support Questions (Read Only)

NiFi: invokehttp retries

umair_ahmed — Mon, 08 May 2017 13:53:49 GMT

Hi..

i am using invokeHttp processor to consume my rest api but whenever 5xx http error retrying back to same processor (invoke Http) but its hitting my infinite time ..i want to limit retry count 5 and it retry at certain time like first retry after 1 min and second after 30 min and vice versa ....thanks in advance

umair

Re: NiFi: invokehttp retries

Wynner — Sun, 18 Aug 2019 01:50:58 GMT

@umair ahmed

You could use the UpdateAttribute and RouteOnAttribute processors to create additional attributes on the flow file that keep track on the number of times the file has looped on a particular relationship. Here is an example of a flow that keep track of every relationship but the original flow file in the InvokeHttp processor:

Here is the configuration of the UpdateAttribute processor

The advanced UI in the RouteOnAttribute can be used to set the delay time using an attribute

Here is the configuration of the RouteOnAttribute processor

The delay is done using the ExecuteStreamCommand processor and calling the sleep command with the appropriate value to wait for the next attempt in using the InvokeHttp processor.

You can adjust for your need.

Re: NiFi: invokehttp retries

Wynner — Mon, 08 May 2017 21:08:54 GMT

@umair ahmed

Here is a template of the above flow.

counter-to-delay-on-loop.xml

Re: NiFi: invokehttp retries

umair_ahmed — Tue, 09 May 2017 12:23:03 GMT

@Wynner

In ExecuteStreamCommand processor command path= ' usr/bin/sleep' what is this thing because i am getting exception in ExecuteStreamCommand processor .."The System cannot find the file specified"

and i am using windows

thanks

umair

Re: NiFi: invokehttp retries

Wynner — Tue, 09 May 2017 19:33:47 GMT

@umair ahmed

That is a command in Unix, to wait the amount of time specified, 30 seconds, one minute whatever is specified. The Windows equivalent is timeout I believe, so just replace the command.

Re: NiFi: invokehttp retries

umair_ahmed — Tue, 09 May 2017 19:48:28 GMT

@Wynner

usr/bin/sleep is seem a directory structure

so i have to replace usr/bin/sleep to timeout only

or something else can you give me an example

thanks

umair

Re: NiFi: invokehttp retries

Wynner — Tue, 09 May 2017 19:57:03 GMT

@umair ahmed

Correct, just replace /usr/bin/sleep with timeout.

What version of Windows are you using?

Re: NiFi: invokehttp retries

umair_ahmed — Tue, 09 May 2017 20:07:54 GMT

@Wynner

windows 10 version 1703

but now after replacing to timeout i am getting error invalid value for timeout(/T) specified .valid range id -1 to 9999

Re: NiFi: invokehttp retries

umair_ahmed — Sun, 18 Aug 2019 01:50:28 GMT

i getting this error now

Re: NiFi: invokehttp retries

umair_ahmed — Tue, 09 May 2017 21:07:44 GMT

@wynner

i am using your template

and now it is firing after every 30 sec for 5 counts

InvokeHTTP Penalty Duration is 30 sec as per my requirement everything is in your template already but it is not working as i want to

Re: NiFi: invokehttp retries

Wynner — Tue, 09 May 2017 22:47:06 GMT

@umair ahmed

I am not following you.

What do you mean by "firing every 30 sec for 5 counts"?

I just used the delay times as en example. How long do you want it to wait each time through?

In my template, iI did the first time 60 seconds, second time 30 minutes, third time 1 hour, fourth time two hours.

Re: NiFi: invokehttp retries

umair_ahmed — Wed, 10 May 2017 00:02:58 GMT

@wynner

i mean by "firing every 30 sec for 5 counts"? that

invokehttp processor Penalty Duration is 30 second so every 30 second it retrying for service (for 5xx error)

and it is retrying 5 times

that is

1st retry after 30 sec

2nd retry after 1 min and so on till 5 count

i want to wait same as you said first time 60 seconds, second time 30 minutes, third time 1 hour, fourth time two hours then break the loop

but in my case it is not working

i think it is problem of windows

Re: NiFi: invokehttp retries

Wynner — Sun, 18 Aug 2019 01:50:20 GMT

@umair ahmed

You are correct, it is a Windows problem. The timeout command does not work with input redirection.

So, use the waitfor command and it will work, here is a snapshot of the reconfigured ExecuteScriptCommand processor:

Even though it works, the waitfor command throws an error because the notanerror event never occurs before the time out period. Also, the sleeptime values have to be in seconds, not minutes or hours.

Re: NiFi: invokehttp retries

umair_ahmed — Wed, 10 May 2017 12:10:27 GMT

@Wynner

not working i am getting error " or ERROR: Invalid syntax. Specify valid numeric value for '/T'. Type "WAITFOR /?" for usage." and i replace sleeptime value to second

Re: NiFi: invokehttp retries

Wynner — Wed, 10 May 2017 19:33:20 GMT

@umair ahmed

What do you have in the property Command Arguments in the ExecuteStreamCommand processor?

Please take a snapshot the Properties tab for the ExecuteStreamCommand processor.

When I said you need to have the value in seconds for the sleeptime attribute, I meant inside the configuration of the advanced UI in the UpdateAttribute processor. In the template I uploaded, I was using minutes.