@Michael Bronson 1) To find the consumer-group related to a topic you can use the below script: for i in `/usr/hdp/current/kafka-broker/bin/zookeeper-shell.sh hdpmaster:2181 ls /consumers 2>&1 | grep consumer | cut -d "[" -f2 | cut -d "]" -f1 | cut -d "," -f1`
do
/usr/hdp/current/kafka-broker/bin/zookeeper-shell.sh hdpmaster:2181 ls /consumers/$i/offsets 2>&1 | grep test
if [ $? == 0 ]
then
echo $i
fi
done 2) Yes, setting the retention.ms using ./bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic my-topic --config retention.ms=1000 3) as @Jordan Moore mentioned there is no automation to delete the topic which is in built to Kafka (we have to follow the process defined above) 4) when I delete a topic we get - Topic <....> is marked for deletion. , is it mean that it will take time until topic will be deleted ? Yes, it does take time to delete a topic and you can also this message if the delete.topic.enable is set to false (it is the default setting) as given previously:Note: to delete a kafka topic we need to set delete.topic.enable=true it requires kafka service to be restarted
... View more
