Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

disconnecting problem between zookeper servers to kafka machines

Highlighted

disconnecting problem between zookeper servers to kafka machines

hi all

we have hadoop cluster - version 2.6.4 , with kafka machines that installed on separated machines

we are facing disconnecting problem between zookeper server to kafka machines

Unable to reconnect to ZooKeeper service

any suggestion how to verify this problem ? and find the reason for this disconecting ?

log from server.log from kafka machine ( under /var/log/kafka )

               at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
[2078-10-07 07:42:13,342] WARN [ReplicaFetcherThread-0-1007], Error in fetch kafka.server.ReplicaFetcherThread$FetchRequest@407ac4e4 (kafka.server.ReplicaFetcherThread)
java.io.IOException: Connection to kafka03.hfge.com:6667 (id: 1007 rack: null) failed
               at kafka.utils.NetworkClientBlockingOps$.awaitReady$1(NetworkClientBlockingOps.scala:83)
               at kafka.utils.NetworkClientBlockingOps$.blockingReady$extension(NetworkClientBlockingOps.scala:93)
               at kafka.server.ReplicaFetcherThread.sendRequest(ReplicaFetcherThread.scala:248)
               at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:238)
               at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:42)
               at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:118)
               at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:103)
               at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
[2078-10-07 07:42:13,551] WARN [ReplicaFetcherThread-0-1002], Error in fetch kafka.server.ReplicaFetcherThread$FetchRequest@2f9f8ecd (kafka.server.ReplicaFetcherThread)
java.io.IOException: Connection to kafka07.hfge.com:6667 (id: 1002 rack: null) failed
               at kafka.utils.NetworkClientBlockingOps$.awaitReady$1(NetworkClientBlockingOps.scala:83)
               at kafka.utils.NetworkClientBlockingOps$.blockingReady$extension(NetworkClientBlockingOps.scala:93)
               at kafka.server.ReplicaFetcherThread.sendRequest(ReplicaFetcherThread.scala:248)
               at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:238)
               at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:42)
               at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:118)
               at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:103)
               at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
[2078-10-07 07:42:15,343] WARN [ReplicaFetcherThread-0-1007], Error in fetch kafka.server.ReplicaFetcherThread$FetchRequest@4dafd0d9 (kafka.server.ReplicaFetcherThread)
java.io.IOException: Connection to kafka03.hfge.com:6667 (id: 1007 rack: null) failed
               at kafka.utils.NetworkClientBlockingOps$.awaitReady$1(NetworkClientBlockingOps.scala:83)
               at kafka.utils.NetworkClientBlockingOps$.blockingReady$extension(NetworkClientBlockingOps.scala:93)
               at kafka.server.ReplicaFetcherThread.sendRequest(ReplicaFetcherThread.scala:248)
               at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:238)
               at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:42)
               at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:118)
               at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:103)
               at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
[2078-10-07 07:42:15,552] WARN [ReplicaFetcherThread-0-1002], Error in fetch kafka.server.ReplicaFetcherThread$FetchRequest@5ce51081 (kafka.server.ReplicaFetcherThread)
java.io.IOException: Connection to kafka07.hfge.com:6667 (id: 1002 rack: null) failed
[2018-09-30 11:28:27,219] WARN Client session timed out, have not heard from server in 20010ms for sessionid 0x2662a207bd60726 (org.apache.zookeeper.ClientCnxn)
[2018-09-30 11:28:58,457] WARN Client session timed out, have not heard from server in 31136ms for sessionid 0x2662a207bd60726 (org.apache.zookeeper.ClientCnxn)
[2018-09-30 11:29:00,255] WARN Unable to reconnect to ZooKeeper service, session 0x2662a207bd60726 has expired (org.apache.zookeeper.ClientCnxn)
Michael-Bronson
Don't have an account?
Coming from Hortonworks? Activate your account here