Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

kafka consumer error in apache metron cluster with 4 node (3 data node and 1 master node)

Highlighted

kafka consumer error in apache metron cluster with 4 node (3 data node and 1 master node)

New Contributor
ERROR Controller 1003 epoch 51 encountered error while electing leader for partition [__consumer_offsets,1] due to: Preferred replica 1002 for partition [__consumer_offsets,1] is either not alive or not in the isr. Current leader and ISR: [{"leader":1004,"leader_epoch":93,"isr":[1004]}]. (state.change.logger) [2019-05-22 23:56:03,878] ERROR Controller 1003 epoch 51 initiated state change for partition [__consumer_offsets,1] from OnlinePartition to OnlinePartition failed (state.change.logger) kafka.common.StateChangeFailedException: encountered error while electing leader for partition [__consumer_offsets,1] due to: Preferred replica 1002 for partition [__consumer_offsets,1] is either not alive or not in the isr. Current leader and ISR: [{"leader":1004,"leader_epoch":93,"isr":[1004]}].        at kafka.controller.PartitionStateMachine.electLeaderForPartition(PartitionStateMachine.scala:368)        at kafka.controller.PartitionStateMachine.kafka$controller$PartitionStateMachine$$handleStateChange(PartitionStateMachine.scala:207)        at kafka.controller.PartitionStateMachine$$anonfun$handleStateChanges$2.apply(PartitionStateMachine.scala:146)        at kafka.controller.PartitionStateMachine$$anonfun$handleStateChanges$2.apply(PartitionStateMachine.scala:145)        at scala.collection.immutable.Set$Set1.foreach(Set.scala:74)        at kafka.controller.PartitionStateMachine.handleStateChanges(PartitionStateMachine.scala:145)        at kafka.controller.KafkaController.onPreferredReplicaElection(KafkaController.scala:665)        at kafka.controller.KafkaController$$anonfun$kafka$controller$KafkaController$$checkAndTriggerPartitionRebalance$4$$anonfun$apply$18$$anonfun$apply$5.apply$mcV$sp(KafkaController.scala:1228)        at kafka.controller.KafkaController$$anonfun$kafka$controller$KafkaController$$checkAndTriggerPartitionRebalance$4$$anonfun$apply$18$$anonfun$apply$5.apply(KafkaController.scala:1223)        at kafka.controller.KafkaController$$anonfun$kafka$controller$KafkaController$$checkAndTriggerPartitionRebalance$4$$anonfun$apply$18$$anonfun$apply$5.apply(KafkaController.scala:1223)        at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:231)        at kafka.controller.KafkaController$$anonfun$kafka$controller$KafkaController$$checkAndTriggerPartitionRebalance$4$$anonfun$apply$18.apply(KafkaController.scala:1220)        at kafka.controller.KafkaController$$anonfun$kafka$controller$KafkaController$$checkAndTriggerPartitionRebalance$4$$anonfun$apply$18.apply(KafkaController.scala:1218)        at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)        at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)        at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226)        at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)        at scala.collection.mutable.HashMap.foreach(HashMap.scala:98)        at kafka.controller.KafkaController$$anonfun$kafka$controller$KafkaController$$checkAndTriggerPartitionRebalance$4.apply(KafkaController.scala:1218)        at kafka.controller.KafkaController$$anonfun$kafka$controller$KafkaController$$checkAndTriggerPartitionRebalance$4.apply(KafkaController.scala:1197)        at scala.collection.immutable.Map$Map4.foreach(Map.scala:181)        at kafka.controller.KafkaController.kafka$controller$KafkaController$$checkAndTriggerPartitionRebalance(KafkaController.scala:1197)        at kafka.controller.KafkaController$$anonfun$onControllerFailover$1.apply$mcV$sp(KafkaController.scala:347)        at kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:110)        at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:56)        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)        at java.lang.Thread.run(Thread.java:745) Caused by: kafka.common.StateChangeFailedException: Preferred replica 1002 for partition [__consumer_offsets,1] is either not alive or not in the isr. Current leader and ISR: [{"leader":1004,"leader_epoch":93,"isr":[1004]}]        at kafka.controller.PreferredReplicaPartitionLeaderSelector.selectLeader(PartitionLeaderSelector.scala:159)        at kafka.controller.PartitionStateMachine.electLeaderForPartition(PartitionStateMachine.scala:345)        ... 31 more