Support Questions
Find answers, ask questions, and share your expertise

Kafka Offset Preserving in Disaster Recovery Architecture

Explorer

Hello, I'm asking about practical solutions for preserving Kafka offset, used by consumers, when failing over between production and disaster recovery Kafka clusters. Had anyone experience something like that before? I read some ideas about using Zookeeper to preserve the Kafka offset of consumers, but I did not try it so far.

-Mahmoud

1 REPLY 1

Expert Contributor

@msabri

Mahmoud, have you looked at Kafka mirrormaker. The process below goes through the steps to setup and failover the mirror.

https://community.hortonworks.com/articles/79891/kafka-mirror-maker-best-practices.html

I have seen this solution implemented in Production.

Graham