Impact of Kafka broker down

[#1]

Hello,

CDH version 5.8, Kafka 2.1.1-1.2.1.p0.18 We have 5 Kafka brokers on production cluster. One of the broker is unavailable due to host issue. 

Is there any impact on running streaming applications, that Kafka was using in real time mode to get message from service.

[#2 · Accepted solution]

Hi @BaluSaiD,

If the topics you're producing/consuming data to/from have at least 2/3 in-sync replicas and min.insync.replicas=2, then it should be ok. If some topics have just 1 replica, and this broker dies, then you will not be able to produce/consume data from this topic.

Properties to keep in mind:

1. Server-side:

min.insync.replicas: When a producer sets acks to "all" (or "-1"), min.insync.replicas specify the minimum number of replicas that must acknowledge a write for the write to be considered successful. If this minimum cannot be met, then the producer will raise an exception (either NotEnoughReplicas or NotEnoughReplicasAfterAppend).
When used together, min.insync.replicas and acks allow you to enforce greater durability guarantees. A typical scenario would be to create a topic with a replication factor of 3, set min.insync.replicas to 2, and produce with acks of "all". This will ensure that the producer raises an exception if a majority of replicas do not receive a write.

2. Producer Side:

ack: The number of acknowledgments the producer requires the leader to have received before considering a request complete. This controls the durability of records that are sent. The following settings are allowed:
acks=0 If set to zero then the producer will not wait for any acknowledgment from the server at all. The record will be immediately added to the socket buffer and considered sent. No guarantee can be made that the server has received the record in this case, and the retries configuration will not take effect (as the client won't generally know of any failures). The offset given back for each record will always be set to -1.
acks=1 This will mean the leader will write the record to its local log but will respond without awaiting full acknowledgment from all followers. In this case, should the leader fail immediately after acknowledging the record but before the followers have replicated it then the record will be lost.
acks=all This means the leader will wait for the full set of in-sync replicas to acknowledge the record. This guarantees that the record will not be lost as long as at least one in-sync replica remains alive. This is the strongest available guarantee. This is equivalent to the acks=-1 setting.

Regards,

Manuel.