Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

How to define topic retention with kafka?

Hello, everybody.

Someone can tell me if it is possible to determine the retention of kafka topics individually and not globally.

In Ambari I have found the "log.retention.hours" option, but it applies to all topics and I would be interested in having a different configuration for the different topics I have.

Greetings

1 ACCEPTED SOLUTION

Super Collaborator

Hello Lanic,

while I have not yet tried to configure it on topic level, this is possible according to the documentation.

Here is the documentation: https://kafka.apache.org/documentation/#topicconfigs

In principal you can use the parameter retention.ms on topic level. It should override the global server settings coming from log.retention.ms for the topic when you provide them. Not sure if retention.hours would be possible as well.

You can set them during the topic creation, or change it later:

bin/kafka-configs.sh --zookeeper localhost:2181  --entity-type topics --entity-name my-topic --alter --add-config retention.ms=128000

View solution in original post

2 REPLIES 2

Super Collaborator

Hello Lanic,

while I have not yet tried to configure it on topic level, this is possible according to the documentation.

Here is the documentation: https://kafka.apache.org/documentation/#topicconfigs

In principal you can use the parameter retention.ms on topic level. It should override the global server settings coming from log.retention.ms for the topic when you provide them. Not sure if retention.hours would be possible as well.

You can set them during the topic creation, or change it later:

bin/kafka-configs.sh --zookeeper localhost:2181  --entity-type topics --entity-name my-topic --alter --add-config retention.ms=128000

Thank you so much for the help, I appreciate it.