@chandramouli muthukumaran Good question. Let me add Apache NiFi as one of your options as it is the "easiest" to implement as you would orchestrate entire data flow with a neat UI. NiFi was design for real time stream simple event processing. So if you use case is within this realm nifi is the way to go. moreover with MiNiFi you can have the process running on the small device (footprint ~40mb) which push data to data center. If you require complex event processing, HDF now comes with storm. so with nifi you get ease of operations, development, data linage, and message guarantee, and highly resilient solution. Oh not to mentioned back pressure when target repository is down. Spark - if you require only complex event processing and can handle microbatching (as little as .5 second) then spark may be good fit.. Spark streaming is in my opinion easier to develop in then storm. No UI Storm - VERY powerful complex stream processing engine with virtually zero latency. Storm now comes with capability to do rolling and tumbling window. Also latest release has back pressuring. No UI. Hope that was helpful to start with.
... View more
