question Re: How to Kafka topic clone. in Archives of Support Questions (Read Only)

How to Kafka topic clone.

Nilesh — Fri, 27 Oct 2017 18:59:57 GMT

Can we clone single Kafka topic? If yes then how?

Re: How to Kafka topic clone.

JordanMoore — Sat, 28 Oct 2017 03:57:37 GMT

At the basics, you would write a producer that consumes from one topic and produces to another.

MirrorMaker is what you are looking for.

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_kafka-component-guide/content/ch_kafka_mirrormaker.html

Re: How to Kafka topic clone.

Nilesh — Sat, 28 Oct 2017 12:13:57 GMT

Thank you for sharing the URL. I am looking for topic cloning within the same cluster. Can we do this.

Re: How to Kafka topic clone.

JordanMoore — Mon, 30 Oct 2017 11:18:15 GMT

Yes, MirrorMaker is not putting a limitation on remote vs local cluster. It is designed for remote clusters because there is almost no need to do it locally. If you are mirroring a topic locally, you must rename it, and if you are going to rename it, then you have consumers/producers using data in both topics?

You are replicating data within the same cluster for little gain while your consumers/producers can easily be configured to use the correct topic(s).