zookeeper log: WARN [/0.0.0.0:3888:QuorumCnxManager@274] - Exception reading or writing challenge: java.net.SocketException: Connection reset 2016-07-14 07:27:24,266 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /40.86.112.11:1336 2016-07-14 07:27:24,457 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@362] - Exception causing close of session 0x0 due to java.io.IOException: Connection reset java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:209) at java.net.SocketInputStream.read(SocketInputStream.java:141) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read(BufferedInputStream.java:265) at java.io.DataInputStream.readInt(DataInputStream.java:387) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:308) storm nimbus log java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at sun.nio.ch.IOUtil.read(IOUtil.java:197) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) at org.apache.storm.thrift.transport.TNonblockingSocket.read(TNonblockingSocket.java:142) at org.apache.storm.thrift.server.AbstractNonblockingServer$FrameBuffer.internalRead(AbstractNonblockingServer.java:539) at org.apache.storm.thrift.server.AbstractNonblockingServer$FrameBuffer.read(AbstractNonblockingServer.java:338) at org.apache.storm.thrift.server.AbstractNonblockingServer$AbstractSelectThread.handleRead(AbstractNonblockingServer.java:203) at org.apache.storm.thrift.server.TNonblockingServer$SelectAcceptThread.select(TNonblockingServer.java:203) at org.apache.storm.thrift.server.TNonblockingServer$SelectAcceptThread.run(TNonblockingServer.java:154) 2016-07-15 04:18:26.065 o.a.s.t.s.AbstractNonblockingServer$FrameBuffer [WARN] Got an IOException in internalRead! ... View more

I have deployed storm cluster with 3 zookeeper node and 1 nimbus 2 supervisor .After running for some time nimbus and supervisor stops and gives java.lang.RuntimeException: org.apache.storm.shade.org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /supervisors at org.apache.storm.util$wrap_in_runtime.invoke(util.clj:54) at org.apache.storm.zookeeper$exists_node_QMARK_$fn__2091.invoke(zookeeper.clj:108) at org.apache.storm.zookeeper$exists_node_QMARK_.invoke(zookeeper.clj:104) at org.apache.storm.zookeeper$mkdirs.invoke(zookeeper.clj:124) at org.apache.storm.cluster_state.zookeeper_state_factory$_mkState$reify__11321.set_ephemeral_node(zookeeper_state_factory.clj:75) at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93) at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28) at org.apache.storm.cluster$mk_storm_cluster_state$reify__3756.supervisor_heartbeat_BANG_(cluster.clj:528) at sun.reflect.GeneratedMethodAccessor30.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93) at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28) at org.apache.storm.daemon.supervisor$fn__9230$exec_fn__2461__auto____9231$heartbeat_fn__9233.invoke(supervisor.clj:790) at org.apache.storm.timer$schedule_recurring$this__6183.invoke(timer.clj:105) at org.apache.storm.timer$mk_timer$fn__6166$fn__6167.invoke(timer.clj:50) at org.apache.storm.timer$mk_timer$fn__6166.invoke(timer.clj:42) at clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.storm.shade.org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /supervisors at org.apache.storm.shade.org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.storm.shade.org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.storm.shade.org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045) at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:226) at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:215) at org.apache.storm.shade.org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:108) at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForegroundStandard(ExistsBuilderImpl.java:212) at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:205) at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:168) at org.apache.storm.shade.org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:39) at org.apache.storm.zookeeper$exists_node_QMARK_$fn__2091.invoke(zookeeper.clj:107) ... 20 more 2016-07-08 11:10:49.100 o.a.s.event [ERROR] Error when processing event java.lang.RuntimeException: org.apache.storm.shade.org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /assignments at org.apache.storm.util$wrap_in_runtime.invoke(util.clj:54) at org.apache.storm.zookeeper$get_children.invoke(zookeeper.clj:187) at org.apache.storm.cluster_state.zookeeper_state_factory$_mkState$reify__11321.get_children(zookeeper_state_factory.clj:127) at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93) at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28) at org.apache.storm.cluster$mk_storm_cluster_state$reify__3756.assignments(cluster.clj:290) at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93) at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28) at org.apache.storm.daemon.supervisor$assignments_snapshot.invoke(supervisor.clj:61) at org.apache.storm.daemon.supervisor$mk_synchronize_supervisor$this__9078.invoke(supervisor.clj:549) at org.apache.storm.event$event_manager$fn__8630.invoke(event.clj:40) at clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.storm.shade.org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /assignments at org.apache.storm.shade.org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.storm.shade.org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.storm.shade.org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1590) at org.apache.storm.shade.org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1625) at org.apache.storm.shade.org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:210) at org.apache.storm.shade.org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:203) at org.apache.storm.shade.org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:108) at org.apache.storm.shade.org.apache.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:200) at org.apache.storm.shade.org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:191) at org.apache.storm.shade.org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:38) at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93) at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28) at org.apache.storm.zookeeper$get_children.invoke(zookeeper.clj:185) ... View more