question Re: How to define topic retention with kafka? in Archives of Support Questions (Read Only)

How to define topic retention with kafka?

xavwebmaster — Mon, 11 Dec 2017 21:04:39 GMT

Hello, everybody.

Someone can tell me if it is possible to determine the retention of kafka topics individually and not globally.

In Ambari I have found the "log.retention.hours" option, but it applies to all topics and I would be interested in having a different configuration for the different topics I have.

Greetings

Re: How to define topic retention with kafka?

arald — Mon, 11 Dec 2017 22:25:38 GMT

Hello Lanic,

while I have not yet tried to configure it on topic level, this is possible according to the documentation.

Here is the documentation: https://kafka.apache.org/documentation/#topicconfigs

In principal you can use the parameter retention.ms on topic level. It should override the global server settings coming from log.retention.ms for the topic when you provide them. Not sure if retention.hours would be possible as well.

You can set them during the topic creation, or change it later:

bin/kafka-configs.sh --zookeeper localhost:2181  --entity-type topics --entity-name my-topic --alter --add-config retention.ms=128000

Re: How to define topic retention with kafka?

xavwebmaster — Mon, 11 Dec 2017 23:00:33 GMT

Thank you so much for the help, I appreciate it.