questions about kafka and compression.type

Jarinek — Fri, 06 May 2022 13:25:21 GMT

Hello,

is it possible to alter compression.type on a kafka topic while having applications on the topic running?

What should be expected hard drive saving factor for text records like json (2KB size) and compression.type=gzip.

In CM, I see a global setting.

Does it apply to KafkaMirrorMaker only?

Are producer and consumer applications somehow affected, when the global parameter changes.

Best regards
Jaro

Re: questions about kafka and compression.type

Babasaheb — Sat, 06 Aug 2022 17:28:10 GMT

@Jarinek

Please check my inline answer to your queries:

1) is it possible to alter compression.type on a kafka topic while having applications on the topic running?
>> Yes, we would recommend stopping the producer and then alert compression type as whenever the Kafka broker gets a compressed batch of messages from a producer. It will decompress the data to validate if the compression type is different at the topic level and producer level and if compression is the same from both sides, then it will write directly to the topic/leader partition.

2) What should be expected hard drive saving factor for text records like json (2KB size) and compression.type=gzip.
>> You should use and test snappy or lz4 for best speed/compression ratio

3) Does it apply to KafkaMirrorMaker only?
>> Yes, as per the screenshot it will apply for the MM only.

4) Are producer and consumer applications somehow affected, when the global parameter changes.
>> If you are changing it from the Kafka side, then it should not affect the applications.

If you found this response assisted with your query, please take a moment to log in and click on KUDOS 🙂 & ”Accept as Solution" below this post.

Thank you.

question Re: questions about kafka and compression.type in Support Questions

questions about kafka and compression.type

Re: questions about kafka and compression.type