NiFi : java.net.Socket.Exception:Broken Pipe (Write Failed) error in InvokeHTTP Processor

[#1]

I am running a NiFi flow where I have InvokeHTTP Processor calling a python script thru flask API. When I am processing many files (in 100's) then some files like (1-2%) fails with java.net.Socket.Exception:Broken Pipe (Write Failed) error in InvokeHTTP Processor. I have observed this happening more when I am running two separate flows accessing same flask API but different python script. What do you think I should change to avoid it. It only happens once in a while for only few files (1-2%) of total flow file. 

[#2]

@vikrant_kumar24    Based on what you describe you are either exceeding the capabilities of the outbound network connection from nifi, or the receiving end API.  I suspect the latter but its hard to tell without seeing the number of nodes and processor scheduling settings. You can try to slow down the execution of invokeHttp to confirm.  If it is set to 0 sec it can create a lot of connections to the endpoint.  Try 5,10,15 seconds and re-evaluate.    Also If you have more than 1 one nifi node, it can create a great deal of concurrent connections.   

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  

Thanks,

Steven