Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Storm Bolt

Storm Bolt

Hello

I have a question regarding Storm. I have devices which are sending data about their actual state (running or not), their location and so on in realtime. Further I have data about the energy consumption of each device in a database. What I would like to do is to join the information of the energy consumption with the device data to compute the actual energy consumption, e.g. in a particular location.

My question: would it be correct to use Kafka to get the data stream from the devices and to use a Storm Bolt to join the data and to compute the energy consumption?

Thank you very much in advance.

Regards,

Daniela

4 REPLIES 4

Re: Storm Bolt

@Daniela S.

Yes, It is possible and this thread can help you with more detail.

this is very much possible. Storm can have multiple spouts. And A bolt consumes any number of input streams, does some processing, and possibly emits new streams.

Re: Storm Bolt

Can you qualify the 'device' a bit more? How small/large is it? Is it a server? A microcontroller performing analog readings in the field? Battery operated? Do you expect to be able to install a Kafka client on this client?

Re: Storm Bolt

Mentor

@Daniela S. has this been resolved? Can you post your solution or accept best answer?

Re: Storm Bolt

Guru

Enriching an event is one of the key reasons CEP tools like storm are necessary. Data coming from devices often does not have all of the details necessary to obtain the potential value from the event. It is often necessary to enrich the event by adding data from some data store. Make sure to load the connection to the store on prepare and to properly size the data store to ensure it will be able to support the required TPS. In general, any kind of store can be used but traditional RDBMS will often be overwhelmed by the volume generated by IOT processing platforms. Key value stores like HBase and if possible, in-memory stores will generally work best for this purpose.