Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kafka Offset Preserving in Disaster Recovery Architecture

Highlighted

Kafka Offset Preserving in Disaster Recovery Architecture

New Contributor

Hello, I'm asking about practical solutions for preserving Kafka offset, used by consumers, when failing over between production and disaster recovery Kafka clusters. Had anyone experience something like that before? I read some ideas about using Zookeeper to preserve the Kafka offset of consumers, but I did not try it so far.

-Mahmoud

1 REPLY 1

Re: Kafka Offset Preserving in Disaster Recovery Architecture

Expert Contributor

@msabri

Mahmoud, have you looked at Kafka mirrormaker. The process below goes through the steps to setup and failover the mirror.

https://community.hortonworks.com/articles/79891/kafka-mirror-maker-best-practices.html

I have seen this solution implemented in Production.

Graham