Kafka configuration issue on Hadoop Bigdata Cluster

Afridi — Wed, 09 Jul 2025 12:54:19 GMT

HI, I have recently configured Kafka services on our Big Data Cluster. However, I am encountering the following issues on the Kafka broker and while establishing Kafka connections:

Issues Observed:
Broker error: Bad : There are 81 offline partitions.

kafka.controller.ControllerEventManager$ControllerEventThread.$anonfun$doWork$1(ControllerEventManager.scala:131)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:31)
at kafka.controller.ControllerEventManager$ControllerEventThread.doWork(ControllerEventManager.scala:131)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)

kafka connect error: [Worker clientId=connect-1, groupId=connect-cluster] Uncaught exception in herder work thread, exiting:
org.apache.kafka.common.errors.TimeoutException: Failed to get offsets by times in 30000ms

Cloudera Bigdata Cluster configuration:

Totol data node 7:
master nodes: 3
and edge node 2

kafka configuration:

Brokers configured on Datanode1, 2 and 3
Gateway selected all nodes
kafka connection on edge1 and edge2

Requesting your assistance in identifying and resolving these errors at your earliest convenience.

Please let me know if you need any specific logs or configuration details.

Re: Kafka configuration issue on Hadoop Bigdata Cluster

DianaTorres — Thu, 10 Jul 2025 01:10:29 GMT

@Afridi Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Kafka experts @rki_ @ChethanYM @paras @Babasaheb who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

question Re: Kafka configuration issue on Hadoop Bigdata Cluster in Support Questions

Kafka configuration issue on Hadoop Bigdata Cluster

Re: Kafka configuration issue on Hadoop Bigdata Cluster