Kafka offsets in DR scenario

3 replies
3512 views
Translate All

Hi All

I am working for a customer to set up a DR environment.

Customer has created a DR as a Separate instance of Cloudera and PROD is a different instance. The replication is schedule-based as of now.

Can we use Kafka production offset location in DR instance? the customer is working on replication of Hive data and Kafka topics. ? will this work out ?

What is the ideal way to restore a streaming job in DR when PROD is down? Any best practices?

Let me know if you need more details.

Thanks in advance

Muthu

Apache Kafka

tarekabouzeid91Answer

Gold ⭐️

You can check Kafka Mirror Maker here Set up MirrorMaker in Cloudera Manager , also if the 2 clusters are secured via kerberos and reside in 2 different realms you need to make sure there's trust between these 2 kerberos realms

tarekabouzeid91Answer

Gold ⭐️

You can check Kafka Mirror Maker here Set up MirrorMaker in Cloudera Manager , also if the 2 clusters are secured via kerberos and reside in 2 different realms you need to make sure there's trust between these 2 kerberos realms

Translate

Nandinin

Gold ⭐️

I believe this is what you are looking for https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/srm-using/topics/srm-migrating-consumer-groups.html

Translate

SME || Kafka | Schema Registry | SMM | SRM

VidyaSargur

Community Manager

@smkmuthu, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. If you are still experiencing the issue, can you provide the information as requested?

Translate

Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.