In storm version apache-storm-1.0.2 , there a field called maxRetries in the retryservice provided for the kafkaSpout and when a bolt fails for > maxretries the message is acked by spout.So does this mean the message them is removed from the kafka queue. If so, with out processing a message we ae actually removing it? Isn't that bad?
The java docs on the KafkaSpout indicate that the fail method for the spout will re-queue the message for playback at a later time
'The tuple emitted by this spout with the msgId identifier has failed to be fully processed. Typically, an implementation of this method will put that message back on the queue to be replayed at a later time.'
but the code itself looks like it just acks the message in order to skip over further processing and no rescheduling takes place. Messages are never "removed" from the Kafka queue per-se, they just won't continue (to attempt) to be processed after it tries so many times unsuccessfully.
http://storm.apache.org/releases/1.0.0/Guaranteeing-message-processing.html.This article talks about removing the message from queue after getting acked.What does removing here mean? Like you have said, if the messages are just not replayed after maxRetries then since it is acked, the message is understood as processed by kafka and so it won't be consumed by another consumer.This leads to missing some messages.How is this dealt in 1.0.2 version of storm