Support Questions

Find answers, ask questions, and share your expertise

What happens when a tuple is further sent to Storm's executor in a state where the queue is full?

Contributor

What happens when a tuple is further sent to Storm's executor in a state where the queue is full?

Block and let sender await? Or a tuple will be dropped?

2 REPLIES 2

Contributor

In this post https://groups.google.com/forum/#!msg/storm-user/c1g_s5L8yuI/Tk8-FgN596sJ I found "Storm will use an unbounded overflow buffer on the spout if it would have been blocked on an emit. Storm will then stop calling nextTuple until it's able to transfer the overflow buffer onto the outgoing queue."...

Contributor

I read this post https://www.mail-archive.com/user@storm.apache.org/msg08232.html and DisruptorQueue.java https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/utils/DisruptorQue...

As that post said, DisruptorQueue has an overflow queue and its type is ConcurrentLinkedQueue, so overflowed messages will be stored in the overflow queue until it ate up memory.