- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Ingest same data in Kafka topic
- Labels:
-
Apache Kafka
Created ‎11-22-2017 02:43 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Team,
What will be happened if we ingest same data, in particular, Kafka topic again? Will it overwrite the data or will make another copy of data or it will just ignore and notify.
Created ‎11-25-2017 02:31 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @Nilesh
There's no relation between data in two different topics. Reading from one topic and writing to another one will copy (and duplicate) data
Created ‎11-25-2017 02:31 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @Nilesh
There's no relation between data in two different topics. Reading from one topic and writing to another one will copy (and duplicate) data
Created ‎11-27-2017 09:21 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @ Abdelkrim Hadjidj
Thank you for the reply. My question is I am ingesting same data again and again on the same topic then what will happen? Will it overwrite the data or will make another copy of data or it will just ignore and notify?
Thank you.
Created ‎11-27-2017 05:49 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Nilesh same answer for one topic. If you send the same data several times to Kafka you will get it several times when you consume.
Created ‎11-27-2017 05:57 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Just look at Kafka as a distibuted log file. even if you insert the same data, again and again, it just appends the data into a distibuted log file.
