Support Questions

Find answers, ask questions, and share your expertise

Kafka MirrorMaker - Mirroring different Kafka versions



We need to use MirrorMaker to replicate data from 2 Kafka clusters running different versions.

Source ------------------------------> Target

Kafka 0.10.1 -> MirrorMaker -> Kafka 0.9.0

We are able to get this to work running MirrorMaker 0.9.0 using the "old zookeeper" consumer type, but would like to know if this is possible with MirrorMaker 0.10.1? We would like to limit the exposure of the source ZK cluster, hence the need to use the new MirrorMaker 0.10.1 consumer. Are there any consumer/producer configs we can use in MirrorMaker to allow us to use version 0.10.1?
# Exception in MirrorMaker when running version 0.10.1
[2017-08-31 16:43:15,799] ERROR Uncaught error in kafka producer I/O thread:  (org.apache.kafka.clients.producer.internals.Sender) org.apache.kafka.common.protocol.types.SchemaException:
Error reading field 'brokers': Error reading field 'host': Error reading string
of length 26992, only 2176 bytes


We either need to figure out how to get MirrorMaker 0.9.0 to use the new client api

OR get MirrorMaker 0.10.1 to use the 0.9.0 compatible message format.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.