Kafka Offset Preserving in Disaster Recovery Architecture

[#1]

Hello, I'm asking about practical solutions for preserving Kafka offset, used by consumers, when failing over between production and disaster recovery Kafka clusters. Had anyone experience something like that before? I read some ideas about using Zookeeper to preserve the Kafka offset of consumers, but I did not try it so far.

-Mahmoud

[#2]

@msabri

Mahmoud, have you looked at Kafka mirrormaker. The process below goes through the steps to setup and failover the mirror.

https://community.hortonworks.com/articles/79891/kafka-mirror-maker-best-practices.html

I have seen this solution implemented in Production.

Graham