https://community.cloudera.com/t5/Support-Questions/Huge-Kafka-consumer-offsets-topic/m-p/393137#M248349 <P>I have a problem with the topic __consumer_offsets, the topic is very large, kafka metrics show that I have 4 billion messages on it, I suspect that message compaction does not work correctly, I constantly see an increase in new messages in the metrics on this topic (however, the total log dir size fluctuates periodically between 19 and 20 GB). What could be the reason that this topic is not being cleaned up?</P> Fri, 06 Sep 2024 08:37:55 GMT Mhl 2024-09-06T08:37:55Z https://community.cloudera.com/t5/Support-Questions/Huge-Kafka-consumer-offsets-topic/m-p/393137#M248349 <P>I have a problem with the topic __consumer_offsets, the topic is very large, kafka metrics show that I have 4 billion messages on it, I suspect that message compaction does not work correctly, I constantly see an increase in new messages in the metrics on this topic (however, the total log dir size fluctuates periodically between 19 and 20 GB). What could be the reason that this topic is not being cleaned up?</P> Fri, 06 Sep 2024 08:37:55 GMT https://community.cloudera.com/t5/Support-Questions/Huge-Kafka-consumer-offsets-topic/m-p/393137#M248349 Mhl 2024-09-06T08:37:55Z https://community.cloudera.com/t5/Support-Questions/Huge-Kafka-consumer-offsets-topic/m-p/393141#M248352 <P>Welcome to the community&nbsp;<a href="https://community.cloudera.com/t5/user/viewprofilepage/user-id/115997">@Mhl</a>.&nbsp;While you wait for a direct response, I did find a <A href="https://community.cloudera.com/t5/Support-Questions/consumer-offsets-partition-size-is-huge/m-p/343557" target="_blank" rel="noopener">similar post</A> that may be of interest.&nbsp;</P> Fri, 06 Sep 2024 12:26:15 GMT https://community.cloudera.com/t5/Support-Questions/Huge-Kafka-consumer-offsets-topic/m-p/393141#M248352 cjervis 2024-09-06T12:26:15Z