Reply
New Contributor
Posts: 1
Registered: ‎11-30-2017

kafka producer lossing data when all brokers are down

 
I am testing below scenario...

producers sends msgs to brokers..
all brokers goes down.. (I bring them down manually)
producers retry, buffers msgs
when brokers are back up
producer sends msgs to brokers

I noticed below...

producer is able to connect back once brokers comes back up and continue sending msgs
but
all those msgs are lost those producers received from source at time when brokers where down

is there a way to retain msgs during brokers down time?
is this correct..... looks like retry only works if brokers are up but fails to acknowledge msgs?


I am using below configurations....

acks=1
buffer.memory=33554432
retries=600
request.timeout.ms=10000
metadata.max.age.ms=30000
retry.backoff.ms=10000

any help is appreciated !!
Expert Contributor
Posts: 152
Registered: ‎07-01-2015

Re: kafka producer lossing data when all brokers are down

Can you share the code? Is it in python?
Announcements