we have 3 kafka's machines in our HDP cluster
we noticed that on each kafka machine kafka disk is almost 100%
( kafka disk represent - /var/kafka/kafka-logs/TOPIC FILES )
/dev/sdd 35T 34T 1T 98% /var/kafka
regrading that we have 3 kafka machines and each kafka disk size is 35T
Please take a look at our Kafka documentation, and refer specifically to the "Log Settings" section:
'log.retention.hours' is probably the setting (along with 'log.segment.bytes') that you will want to read up on, that will reduce your retention time period.