question Re: Ingest same data in Kafka topic in Archives of Support Questions (Read Only)

Ingest same data in Kafka topic

Nilesh — Wed, 22 Nov 2017 22:43:41 GMT

Team,

What will be happened if we ingest same data, in particular, Kafka topic again? Will it overwrite the data or will make another copy of data or it will just ignore and notify.

Re: Ingest same data in Kafka topic

ahadjidj — Sat, 25 Nov 2017 22:31:38 GMT

Hi @Nilesh

There's no relation between data in two different topics. Reading from one topic and writing to another one will copy (and duplicate) data

Re: Ingest same data in Kafka topic

Nilesh — Mon, 27 Nov 2017 17:21:20 GMT

Hi @ Abdelkrim Hadjidj

Thank you for the reply. My question is I am ingesting same data again and again on the same topic then what will happen? Will it overwrite the data or will make another copy of data or it will just ignore and notify?

Thank you.

Re: Ingest same data in Kafka topic

ahadjidj — Tue, 28 Nov 2017 01:49:37 GMT

@Nilesh same answer for one topic. If you send the same data several times to Kafka you will get it several times when you consume.

Re: Ingest same data in Kafka topic

kramalingam — Tue, 28 Nov 2017 01:57:16 GMT

Just look at Kafka as a distibuted log file. even if you insert the same data, again and again, it just appends the data into a distibuted log file.