Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Nifi Site to Site - Broken pipe error

Nifi Site to Site - Broken pipe error

Hi All,

I have setup nifi site to site. I am sending data from various nifi standalone instances to a single nifi cluster via site to site.

After sending data for sometime, i see below errors

"Connection reset by peer" and "Broken Pipe"

2018-04-20 09:47:28,366 ERROR [Timer-Driven Process Thread-4] o.a.nifi.remote.StandardRemoteGroupPort RemoteGroupPort[name=InputPortName,targets=http://HostIP:62200/nifi] failed to communicate with remote NiFi instance due to org.apache.nifi.processor.exception.ProcessException: IOException thrown from RemoteGroupPort[name=InputPortName,targets=http://HostIP:62200/nifi]: java.io.IOException: Failed to send data to Peer[url=http://Hostname1:62200/nifi-api] due to java.io.IOException: Pipe closed 2018-04-20 09:47:28,394 ERROR [I/O dispatcher 137217641] o.a.n.r.util.SiteToSiteRestApiClient Failed to send data to http://Hostname3:62200/nifi-api/data-transfer/input-ports/066c4cc3-f222-1b0f-0000-0000123ddd0d/trans... due to java.io.IOException: Connection reset by peer java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.write0(Native Method) at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) at sun.nio.ch.IOUtil.write(IOUtil.java:65) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471) at org.apache.http.impl.nio.reactor.SessionOutputBufferImpl.flush(SessionOutputBufferImpl.java:166) at org.apache.http.impl.nio.DefaultNHttpClientConnection.produceOutput(DefaultNHttpClientConnection.java:297) at org.apache.http.impl.nio.client.InternalIODispatch.onOutputReady(InternalIODispatch.java:86) at org.apache.http.impl.nio.client.InternalIODispatch.onOutputReady(InternalIODispatch.java:39) at org.apache.http.impl.nio.reactor.AbstractIODispatch.outputReady(AbstractIODispatch.java:145) at org.apache.http.impl.nio.reactor.BaseIOReactor.writable(BaseIOReactor.java:188) at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:341) at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588) at java.lang.Thread.run(Thread.java:748)

Please can someone help me out?

1 REPLY 1
Highlighted

Re: Nifi Site to Site - Broken pipe error

@Bharath Phatak

I am also getting the same error some times while transferring files using NiFi S2S. Did you find any solution for this?

2019-01-16 12:18:12,566 ERROR [I/O dispatcher 8585] o.a.n.r.util.SiteToSiteRestApiClient Failed to send data to http://hdfnifiprod01.medassurant.local:9090/nifi-api/data-transfer/input-ports/3e9238a3-0168-1000-ff... due to java.net.SocketTimeoutException
java.net.SocketTimeoutException: null
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:376)
        at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:92)
        at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:39)
        at org.apache.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:263)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReactor.java:492)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:213)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
        at java.lang.Thread.run(Thread.java:745)
2019-01-16 12:18:12,570 ERROR [Timer-Driven Process Thread-3] o.a.nifi.remote.StandardRemoteGroupPort RemoteGroupPort[name=FromPhProd,targets=http://hdfnifiprod01.medassurant.local:9090/nifi/,http://hdfnifiprod02.medassurant.local:9090/nifi/] failed to communicate with remote NiFi instance due to org.apache.nifi.processor.exception.ProcessException: IOException thrown from RemoteGroupPort[name=FromPhProd,targets=http://hdfnifiprod01.medassurant.local:9090/nifi/,http://hdfnifiprod02.medassurant.local:9090/nifi/]: java.io.IOException: Failed to send data to Peer[url=http://hdfnifiprod01.medassurant.local:9090/nifi-api] due to java.io.IOException: Pipe closed

Don't have an account?
Coming from Hortonworks? Activate your account here