Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to define topic retention with kafka?

avatar
Contributor

Hello, everybody.

Someone can tell me if it is possible to determine the retention of kafka topics individually and not globally.

In Ambari I have found the "log.retention.hours" option, but it applies to all topics and I would be interested in having a different configuration for the different topics I have.

Greetings

1 ACCEPTED SOLUTION

avatar
Super Collaborator

Hello Lanic,

while I have not yet tried to configure it on topic level, this is possible according to the documentation.

Here is the documentation: https://kafka.apache.org/documentation/#topicconfigs

In principal you can use the parameter retention.ms on topic level. It should override the global server settings coming from log.retention.ms for the topic when you provide them. Not sure if retention.hours would be possible as well.

You can set them during the topic creation, or change it later:

bin/kafka-configs.sh --zookeeper localhost:2181  --entity-type topics --entity-name my-topic --alter --add-config retention.ms=128000

View solution in original post

2 REPLIES 2

avatar
Super Collaborator

Hello Lanic,

while I have not yet tried to configure it on topic level, this is possible according to the documentation.

Here is the documentation: https://kafka.apache.org/documentation/#topicconfigs

In principal you can use the parameter retention.ms on topic level. It should override the global server settings coming from log.retention.ms for the topic when you provide them. Not sure if retention.hours would be possible as well.

You can set them during the topic creation, or change it later:

bin/kafka-configs.sh --zookeeper localhost:2181  --entity-type topics --entity-name my-topic --alter --add-config retention.ms=128000

avatar
Contributor

Thank you so much for the help, I appreciate it.