Support Questions

Find answers, ask questions, and share your expertise

Kafka configuration issue on Hadoop Bigdata Cluster

avatar
Frequent Visitor

HI, I have recently configured Kafka services on our Big Data Cluster. However, I am encountering the following issues on the Kafka broker and while establishing Kafka connections:

Issues Observed:
Broker error:  Bad : There are 81 offline partitions.

kafka.controller.ControllerEventManager$ControllerEventThread.$anonfun$doWork$1(ControllerEventManager.scala:131)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
  at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:31)
  at kafka.controller.ControllerEventManager$ControllerEventThread.doWork(ControllerEventManager.scala:131)
  at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)

kafka connect error: [Worker clientId=connect-1, groupId=connect-cluster] Uncaught exception in herder work thread, exiting: 
org.apache.kafka.common.errors.TimeoutException: Failed to get offsets by times in 30000ms

Cloudera Bigdata Cluster configuration:

Totol data node 7:
master nodes: 3
and edge node 2

kafka configuration:

Brokers configured on Datanode1, 2 and 3
Gateway selected all nodes
kafka connection on edge1 and edge2

Requesting your assistance in identifying and resolving these errors at your earliest convenience.

Please let me know if you need any specific logs or configuration details.

1 REPLY 1

avatar
Community Manager

@Afridi Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Kafka experts @rki_ @ChethanYM @paras @Babasaheb who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.


Regards,

Diana Torres,
Senior Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community: