Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

kafka parameters for retention and purging

Highlighted

kafka parameters for retention and purging

we have 3 kafka's machines in our HDP cluster


we noticed that on each kafka machine kafka disk is almost 100%

( kafka disk represent - /var/kafka/kafka-logs/TOPIC FILES )

/dev/sdd 35T 34T 1T 98%  /var/kafka

regrading that we have 3 kafka machines and each kafka disk size is 35T

what are the ambari kafka parameters that should be tune or to perform "retention purging" ?
Michael-Bronson
1 REPLY 1

Re: kafka parameters for retention and purging

Expert Contributor

Please take a look at our Kafka documentation, and refer specifically to the "Log Settings" section:

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.5/bk_kafka-component-guide/content/kafka-brok...

'log.retention.hours' is probably the setting (along with 'log.segment.bytes') that you will want to read up on, that will reduce your retention time period.