questions about kafka and compression.type

[#1]

Hello,

is it possible to alter compression.type on a kafka topic while having applications on the topic running?

What should be expected hard drive saving factor for text records like json (2KB size) and compression.type=gzip.

In CM, I see a global setting.

Does it apply to KafkaMirrorMaker only?

Are producer and consumer applications somehow affected, when the global parameter changes.

Best regards
Jaro

[#2 · Accepted solution]

@Jarinek 

Please check my inline answer to your queries: 

1) is it possible to alter compression.type on a kafka topic while having applications on the topic running?
>> Yes, we would recommend stopping the producer and then alert compression type as whenever the Kafka broker gets a compressed batch of messages from a producer. It will decompress the data to validate if the compression type is different at the topic level and producer level and if compression is the same from both sides, then it will write directly to the topic/leader partition. 

2) What should be expected hard drive saving factor for text records like json (2KB size) and compression.type=gzip.
>> You should use and test snappy or lz4 for best speed/compression ratio

3) Does it apply to KafkaMirrorMaker only?
>> Yes, as per the screenshot it will apply for the MM only.

4) Are producer and consumer applications somehow affected, when the global parameter changes.
>> If you are changing it from the Kafka side, then it should not affect the applications.

If you found this response assisted with your query, please take a moment to log in and click on  KUDOS 🙂 & ”Accept as Solution" below this post.

Thank you.