Cloudera Community
mike_bronson7
user rank
Guru

Re: how to force delete topic from kafka

by Guru in Support Questions
‎12-02-2017 07:00 PM
‎12-02-2017 07:00 PM
@Jordan - regrading what you said purge the Zookeeper records : do you mean to delete the topic by rmr /brokers/topics/hgpo.llo.prmt.processed from the Zc server ... View more

Re: what is the safe and best way to delete the ka...

by Guru in Support Questions
‎12-02-2017 05:32 PM
‎12-02-2017 05:32 PM
another thing the topics are exist under /brokers/topics/* and not under /consumers/..../.. so in this case I gues we need to remove the topic as rmr /brokers/topics/hgpo.llo.prmt.processed , can you confirm this please ... View more

Re: what is the safe and best way to delete the ka...

by Guru in Support Questions
‎12-02-2017 04:13 PM
‎12-02-2017 04:13 PM
just to be sure , so it will be safe to remove the topics use rm-rf as ( rm -rf hgpo.llo.prmt.processed-28 . rm -rf hgpo.llo.prmt.processed-28 and so on on all nodes ) ? ... View more

how to force delete topic from kafka

by Guru in Support Questions
‎12-01-2017 11:17 AM
‎12-01-2017 11:17 AM
we do the following steps in order to delete the topic - hgpo.llo.prmt.processed but even after 12 hours topics folders still not deleted from /var/kafka/kafka-logs note - we set - delete.topic.enable=true hkafka01 kafka-logs]# /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper master01:2181 --alter --topic hgpo.llo.prmt.processed--config retention.ms=1000 WARNING: Altering topic configuration from this script has been deprecated and may be removed in future releases. Going forward, please use kafka-configs.sh for this functionality Updated config for topic "hgpo.llo.prmt.processedd" kafka01 kafka-logs]# /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper master01:2181 --delete --topic hgpo.llo.prmt.processed Topic hgpo.llo.prmt.processed is already marked for deletion. how to force the topic deletion ? on kafka machine ( production machine ) , we see that: ( no free space ) /dev/sdb 15500450528 15500434144 0 100% /var/kafka and under /var/kafka/kafka-logs , we still see all topic folders (huge size) as example: ( down ) so is it possible just to remove these folders as example - by rm -rf hgpo.llo.prmt.processed-28 ? under /var/kafka/kafka-logs we have many topic folders as : 117G hgpo.llo.prmt.processed-28 117G hgpo.llo.prmt.processed-29 117G hgpo.llo.prmt.processed-3 117G hgpo.llo.prmt.processed-30 117G hgpo.llo.prmt.processed-31 117G hgpo.llo.prmt.processed-32 . . . . . ... View more

Re: what is the safe and best way to delete the ka...

by Guru in Support Questions
‎12-01-2017 08:31 AM
‎12-01-2017 08:31 AM
@Venkata thank you , I run the script with the topic-name (hgpo.llo.prmt.processed) instead test , and consumer-group not found , I mean no output from the script , is is possible ? second as you know we get Topic mcapi.avro.pri.processed is already marked for deletion after we run "delete kafka topic" and we run it yesterday - why it take so along time ? , ( in my kafka - delete.topic.enable=true is set by default) so in that case can we remove the topic under /var/kafka/kafka-logs by rm -rf "topic name " ? as rm -rf hgpo.llo.prmt.processed-28 rm -rf hgpo.llo.prmt.processed-29 and so on for all other topics 117G hgpo.llo.prmt.processed-3 117G hgpo.llo.prmt.processed-30 117G hgpo.llo.prmt.processed-31 117G hgpo.llo.prmt.processed-32 . . . ... View more

Re: what is the safe and best way to delete the ka...

by Guru in Support Questions
‎11-30-2017 09:36 PM
‎11-30-2017 09:36 PM
just one note , when I delete a topic we get - Topic <....> is marked for deletion. , is it mean that it will take time until topic will be deleted ? ... View more

Re: replace faulty disk on the worker machine

by Guru in Support Questions
‎11-30-2017 07:05 PM
‎11-30-2017 07:05 PM
thank you for the answer , so can we summarize the steps 1. shutdown the worker machine ( stop all components ) 2. replace the faulty disk with new disk ( same size ) 3. start up the worker machine 4. create ext4 file system on the new disk - sdb (by mkfs) 5. start all worker component ( it will create the relevant folders under the sdb disk please let me know if my steps are true second - about "Ambari to put this machine into a maintenance state" - how to set it on the ambari ? and on which of my steps need to defined this ? ... View more

replace faulty disk on the worker machine

by Guru in Support Questions
‎11-30-2017 06:03 PM
‎11-30-2017 06:03 PM
we have worker machine in the ambari cluster that have faulty disk disk device name is /dev/sdb so we need to replace the faulty disk with a new disk what are the steps that need to do on the worker machine before replacing the disk and after replacing the disk ? ... View more
Labels:

Re: what is the safe and best way to delete the ka...

by Guru in Support Questions
‎11-30-2017 05:20 PM
‎11-30-2017 05:20 PM
about my question - what is the mechanizem that suppose to delete automatically the topics ? , I not sure about it but what I am asking is about auto deletion process that remove the topic from the folder , is it truely defined somewhwhere ? ) ... View more

Re: what is the safe and best way to delete the ka...

by Guru in Support Questions
‎11-30-2017 05:14 PM
‎11-30-2017 05:14 PM
can you give more details about the "1) Make sure to set the message retention time of a topic to 1000ms (1s) to stop the inflow --> using retention.ms (it does take some time to delete all logs to free up log segments" , can you please described it step by step , or maybe do you mean that I need to run the CLI as : ./bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic my-topic --config retention.ms=1000 ?? ... View more
Contact Me
Online
Offline
Last Visited
‎08-27-2024 09:17 AM
Community Statistics
Member Since ‎08-08-2017 09:40 AM
Last Visited ‎08-27-2024 09:17 AM
Posts 1,652
Kudos received 29