- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
In Apache NIFI requests are queued and showing socket timeout exception ,not getting processed
- Labels:
-
Apache NiFi
Created 11-08-2023 01:37 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
1000 requests are queued in NIFI , invoke HTTP processor is showing socket timeout exception , i have added a self loop in invoke http processor so that it keep on retrying . These requests are stuck permanently.
My issue is endpoint which invoke http processor is trying to invoke is up and running . i checked from postman i am able to get success response.
2023-11-08 12:17:34,783 ERROR [Timer-Driven Process Thread-138] o.a.nifi.processors.standard.InvokeHTTP InvokeHTTP[id=b48ff3b8-befb-32c2-b670-83445e1873f2] Routing to Failure due to exception: java.net.SocketTimeoutException: connect timed out: java.net.SocketTimeoutException: connect timed out
java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:476)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:218)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:394)
at java.net.Socket.connect(Socket.java:606)
at okhttp3.internal.platform.Platform.connectSocket(Platform.java:124)
at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:223)
at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:149)
at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:195)
at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:121)
Created 11-08-2023 07:36 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Rohit1997jio If your endpoint test works with Postman, and your invokeHttp is setup similary, the above error suggest the nifi node cannot connect to the endpoint. You would need to ensure you have network connectivity from nifi host to endpoint.
Created 11-08-2023 09:29 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
i created a new flow with same end points , some requests are getting successful , but still few requests are still getting stuck in invoke http processor giving socket timeout exception
Created 11-13-2023 07:03 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Sometimes I need to increase the timeouts in the processor configurations. You may also need to reduce the execution time for the processor incase the endpoint cannot handle too many requests at once.
Created 11-08-2023 07:36 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Rohit1997jio If your endpoint test works with Postman, and your invokeHttp is setup similary, the above error suggest the nifi node cannot connect to the endpoint. You would need to ensure you have network connectivity from nifi host to endpoint.
Created 11-08-2023 09:29 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
i created a new flow with same end points , some requests are getting successful , but still few requests are still getting stuck in invoke http processor giving socket timeout exception
Created 11-13-2023 07:03 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Sometimes I need to increase the timeouts in the processor configurations. You may also need to reduce the execution time for the processor incase the endpoint cannot handle too many requests at once.
Created 07-17-2024 07:22 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Did you find any solution about this issue ? I have exactly the same problem.
An invoke HTTP get a time out on the flow file. A retry is implemented and the flow file get stucked with a timeout each time.
Works fine with same configuration to call API with postman and a curl from server nifi node.
The only way is to restart the invoke http processor and after few hours and some flow files a new occurence appears.
Error :
2024-07-17 14:55:06,783 ERROR [Timer-Driven Process Thread-17] o.a.nifi.processors.standard.InvokeHTTP InvokeHTTP[id=ba9d2024-8df1-323d-b0ba-54402272b188] Routing to Failure due to exception: java.net.SocketTimeoutException: timeout: java.net.SocketTimeoutException: timeout
java.net.SocketTimeoutException: timeout
at okhttp3.internal.http2.Http2Stream$StreamTimeout.newTimeoutException(Http2Stream.java:593)
at okhttp3.internal.http2.Http2Stream$StreamTimeout.exitAndThrowIfTimedOut(Http2Stream.java:601)
at okhttp3.internal.http2.Http2Stream$FramingSink.emitFrame(Http2Stream.java:510)
at okhttp3.internal.http2.Http2Stream$FramingSink.close(Http2Stream.java:538)
at okio.RealBufferedSink.close(RealBufferedSink.java:236)
at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:63)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:185)
at okhttp3.RealCall.execute(RealCall.java:69)
at org.apache.nifi.processors.standard.InvokeHTTP.onTrigger(InvokeHTTP.java:850)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1174)
at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Thanks