Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

OutputCollector in storm is creating new streams rather than throwing exception in storm 1.0.x.

Highlighted

OutputCollector in storm is creating new streams rather than throwing exception in storm 1.0.x.

In our storm component we define list of streams and associate them to next set of bolts on creation of topology.And on actual tuple incoming we emit to respective stream and if stream is not already defined then emit method should throw exception as next set of bolts are not defined for this unknown stream.This was working fine till Storm0.10.0.But after moving to storm 1.0.1. emit method is created new stream if tuple is coming for unknown stream.

Is there any config to not to create new streams for above condition(i.e have the same behavior as of 0.10.0)?

Is there anything changed in storm1.0.x specific to this behavior[I can't see in release notes or may be i missed]

3 REPLIES 3
Highlighted

Re: OutputCollector in storm is creating new streams rather than throwing exception in storm 1.0.x.

Contributor

What is the use case for throwing an exception? Are you catching it and doing something with it? The best thing to do is just fix the topology code so that it does not emit to a stream that is not created.

Highlighted

Re: OutputCollector in storm is creating new streams rather than throwing exception in storm 1.0.x.

The use case is: I am getting Json message from a spout and then BoltA is processing that message and emit to a stream and there is listener bolt from set B which is saving that message to database.I was emitting the message without checking if stream already exists or not .So earlier(Storm 0.10.0) Internal Storm code was throwing exception for this undefined stream.But now with Storm 1.0.1 rather than throwing exception it is creating new stream(confirmed the same from Storm UI) and message is not being saved as there is no predefined listener bolt for this stream.

Although i added precheck in my code to not to emit. But still question is: Is there any config to disable creation of stream for undefined data.

Highlighted

Re: OutputCollector in storm is creating new streams rather than throwing exception in storm 1.0.x.

Rising Star

Storm will discard messages that have no downstream subscribers (bolts), are you experiencing issues with back pressure or topology failures?

Don't have an account?
Coming from Hortonworks? Activate your account here