Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to define topic retention with kafka?

Solved Go to solution
Highlighted

How to define topic retention with kafka?

New Contributor

Hello, everybody.

Someone can tell me if it is possible to determine the retention of kafka topics individually and not globally.

In Ambari I have found the "log.retention.hours" option, but it applies to all topics and I would be interested in having a different configuration for the different topics I have.

Greetings

1 ACCEPTED SOLUTION

Accepted Solutions

Re: How to define topic retention with kafka?

Super Collaborator

Hello Lanic,

while I have not yet tried to configure it on topic level, this is possible according to the documentation.

Here is the documentation: https://kafka.apache.org/documentation/#topicconfigs

In principal you can use the parameter retention.ms on topic level. It should override the global server settings coming from log.retention.ms for the topic when you provide them. Not sure if retention.hours would be possible as well.

You can set them during the topic creation, or change it later:

bin/kafka-configs.sh --zookeeper localhost:2181  --entity-type topics --entity-name my-topic --alter --add-config retention.ms=128000
2 REPLIES 2

Re: How to define topic retention with kafka?

Super Collaborator

Hello Lanic,

while I have not yet tried to configure it on topic level, this is possible according to the documentation.

Here is the documentation: https://kafka.apache.org/documentation/#topicconfigs

In principal you can use the parameter retention.ms on topic level. It should override the global server settings coming from log.retention.ms for the topic when you provide them. Not sure if retention.hours would be possible as well.

You can set them during the topic creation, or change it later:

bin/kafka-configs.sh --zookeeper localhost:2181  --entity-type topics --entity-name my-topic --alter --add-config retention.ms=128000

Re: How to define topic retention with kafka?

New Contributor

Thank you so much for the help, I appreciate it.

Don't have an account?
Coming from Hortonworks? Activate your account here