Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

how to force delete topic from kafka

avatar

we do the following steps in order to delete the topic - hgpo.llo.prmt.processed

but even after 12 hours topics folders still not deleted from /var/kafka/kafka-logs

note - we set - delete.topic.enable=true

hkafka01 kafka-logs]# /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper master01:2181 --alter --topic hgpo.llo.prmt.processed--config retention.ms=1000

WARNING: Altering topic configuration from this script has been deprecated and may be removed in future releases. Going forward, please use kafka-configs.sh for this functionality Updated config for topic "hgpo.llo.prmt.processedd"

kafka01 kafka-logs]# /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper master01:2181 --delete --topic hgpo.llo.prmt.processed 

Topic hgpo.llo.prmt.processed is already marked for deletion.

how to force the topic deletion ?

on kafka machine ( production machine ) , we see that: ( no free space )

/dev/sdb                 15500450528 15500434144         0 100% /var/kafka

and under /var/kafka/kafka-logs , we still see all topic folders (huge size) as example: ( down )

so is it possible just to remove these folders as example - by rm -rf hgpo.llo.prmt.processed-28 ?

under /var/kafka/kafka-logs we have many topic folders as :

117G hgpo.llo.prmt.processed-28 
117G hgpo.llo.prmt.processed-29
117G hgpo.llo.prmt.processed-3
117G hgpo.llo.prmt.processed-30
117G hgpo.llo.prmt.processed-31
117G hgpo.llo.prmt.processed-32
.
.
.
.
.

Michael-Bronson
1 ACCEPTED SOLUTION

avatar
Super Collaborator

Is topic deletion enabled at the broker level (delete.topic.enable=true) in the entire cluster, and did you restart it if you did enable it?

Maybe since the disk is full, Kafka and related services are refusing to start. Have you verified there are such processes running on the machine?

The nuclear, manual option would be to delete the topic data from the broker, but you must also purge the Zookeeper records for this topic

View solution in original post

2 REPLIES 2

avatar
Super Collaborator

Is topic deletion enabled at the broker level (delete.topic.enable=true) in the entire cluster, and did you restart it if you did enable it?

Maybe since the disk is full, Kafka and related services are refusing to start. Have you verified there are such processes running on the machine?

The nuclear, manual option would be to delete the topic data from the broker, but you must also purge the Zookeeper records for this topic

avatar
@Jordan - regrading what you said purge the Zookeeper records :
do you mean to delete the topic by rmr /brokers/topics/hgpo.llo.prmt.processed from the Zc server
Michael-Bronson