Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kafka alter topic requires broker restart to take effect

Kafka alter topic requires broker restart to take effect

Explorer

Version Info:

 

Kafka Version: 0.8.2

Cloudera Enterprise: 5.4.7

 

Problem:

 

We have a topic with following configuration for cleanup:

 

min.cleanable.dirty.ratio=0.01,segment.ms=3600000,min.insync.replicas=2,cleanup.policy=compact

 

We changed the topic cleanup policy to delete by issuing an alter topic as follows:

 

kafka-topics --alter --zookeeper <zk quorum> --config retention.ms=2592000000 --delete-config min.cleanable.dirty.ratio --delete-config segment.ms --delete-config cleanup.policy --topic <topic_name>

 

This removed all those configs and config after alter looked like this:

retention.ms=2592000000,min.insync.replicas=2

 

Since default cleanup policy is delete, we just deleted it instead of changing to delete explicitly.

 

Our log cleaner thread runs every 5 min. (In cloudera manager config page). But these changes are not taking effect until i rolling restarted our Kafka broker.

 

I dont see any reference to restart in kafka alter documentation. I am not sure why it is not taking effect immediately.

Are there any other config values that govern (apart from cleaner thread schedule) that governs when and how cleanup policy takes effect ?

 

Thanks,

Vivek

 

3 REPLIES 3

Re: Kafka alter topic requires broker restart to take effect

Contributor
Restart of kafka broker can make the change effective

Re: Kafka alter topic requires broker restart to take effect

Explorer

Thanks for the feedback. That's what we did and it applied the changes immediately.

Highlighted

Re: Kafka alter topic requires broker restart to take effect

New Contributor

Hi,

      Is this still a requirement in 0.9.0.1 or 0.10 of kafka release?  We are currently on 0.9.0.1 but may upgrade to 0.10 soon. Restart is not a feasible option for us for altering retention period for a topic.

 

Regards

Swami