Skip to main content
Rohit1997jio
Rohit1997jio
Silver ⭐️⭐️⭐️
November 8, 2023
Solved

In Apache NIFI requests are queued and showing socket timeout exception ,not getting processed

1000 requests are queued in NIFI , invoke HTTP processor is showing socket timeout exception , i have added a self loop in invoke http processor so that it keep on retrying . These requests are stuck permanently.

My issue is endpoint which invoke http processor is trying to invoke is up and running .  i checked from postman i am able to get success response.

 

2023-11-08 12:17:34,783 ERROR [Timer-Driven Process Thread-138] o.a.nifi.processors.standard.InvokeHTTP InvokeHTTP[id=b48ff3b8-befb-32c2-b670-83445e1873f2] Routing to Failure due to exception: java.net.SocketTimeoutException: connect timed out: java.net.SocketTimeoutException: connect timed out
java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:476)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:218)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:394)
at java.net.Socket.connect(Socket.java:606)
at okhttp3.internal.platform.Platform.connectSocket(Platform.java:124)
at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:223)
at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:149)
at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:195)
at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:121)

steven-matison
steven-matisonAnswer
Platinum ⭐️⭐️
November 13, 2023

Sometimes I need to increase the timeouts in the processor configurations.    You may also need to reduce the execution time for the processor incase the endpoint cannot handle too many requests at once.

View in thread

4 replies

steven-matison
Platinum ⭐️⭐️
steven-matison
Platinum ⭐️⭐️
November 8, 2023

@Rohit1997jio  If your endpoint test works with Postman, and your invokeHttp is setup similary, the above error suggest the nifi node cannot connect to the endpoint.  You would need to ensure you have network connectivity from nifi host to endpoint.   

Translate
Rohit1997jio
Rohit1997jioAuthor
Silver ⭐️⭐️⭐️
November 8, 2023

i created a new flow with same end points , some requests are getting successful , but still few requests are still getting stuck in invoke http processor giving socket timeout exception

Translate
steven-matison
Platinum ⭐️⭐️
steven-matisonAnswer
Platinum ⭐️⭐️
November 13, 2023

Sometimes I need to increase the timeouts in the processor configurations.    You may also need to reduce the execution time for the processor incase the endpoint cannot handle too many requests at once.

Translate
dupuy_gregory
dupuy_gregory
Gold ⭐️
July 17, 2024

Did you find any solution about this issue ? I have exactly the same problem.

An invoke HTTP get a time out on the flow file. A retry is implemented and the flow file get stucked with a timeout each time.

Works fine with same configuration to call API with postman and a curl from server nifi node.

The only way is to restart the invoke http processor and after few hours and some flow files a new occurence appears.

Error :

2024-07-17 14:55:06,783 ERROR [Timer-Driven Process Thread-17] o.a.nifi.processors.standard.InvokeHTTP InvokeHTTP[id=ba9d2024-8df1-323d-b0ba-54402272b188] Routing to Failure due to exception: java.net.SocketTimeoutException: timeout: java.net.SocketTimeoutException: timeout
java.net.SocketTimeoutException: timeout
at okhttp3.internal.http2.Http2Stream$StreamTimeout.newTimeoutException(Http2Stream.java:593)
at okhttp3.internal.http2.Http2Stream$StreamTimeout.exitAndThrowIfTimedOut(Http2Stream.java:601)
at okhttp3.internal.http2.Http2Stream$FramingSink.emitFrame(Http2Stream.java:510)
at okhttp3.internal.http2.Http2Stream$FramingSink.close(Http2Stream.java:538)
at okio.RealBufferedSink.close(RealBufferedSink.java:236)
at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:63)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:185)
at okhttp3.RealCall.execute(RealCall.java:69)
at org.apache.nifi.processors.standard.InvokeHTTP.onTrigger(InvokeHTTP.java:850)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1174)
at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)

 

Thanks

Translate
Terms & ConditionsAccessibility statement