Reply
Explorer
Posts: 19
Registered: ‎04-23-2019

How kafka survives data loss and data duplication in a cluster?

 
Posts: 1,894
Kudos: 433
Solutions: 303
Registered: ‎07-31-2013

Re: How kafka survives data loss and data duplication in a cluster?

Have you gone over Kafka documentation? Are there specific parts or scenarios beyond the ones mentioned that you have these questions about?

> data loss

Kafka provides topic partition replication.

> data duplication

Kafka does not do anything specific for deduplicating data. Assuming you're asking about exactly-once processing semantics, it depends on your application and how it leverages Kafka. One record of this is at https://www.confluent.io/blog/exactly-once-semantics-are-possible-heres-how-apache-kafka-does-it/
Announcements
New solutions