Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

spark vs flink vs storm

Highlighted

spark vs flink vs storm

Expert Contributor

what is the difference between spark, flink and storm. how to choose one of them or a particular use case ? can anyone give use case this components of hadoop.

4 REPLIES 4

Re: spark vs flink vs storm

Rising Star

Technical:

Spark Vs Storm can be decided based on amount of branching you have in your pipeline. Storm can handle complex branching whereas it's very difficult to do so with Spark. Branching means if you have events/messages divided into streams of different types based on some criteria. This is made possible by the fact that Storm operates on a per event basis whereas Spark operates on batches. So if have branching or reason to operate on a per event basis Storm should be your choice.

If you have a linear pipeline, something like validate->transform->ingest then you can perform Apples to Apples comparison as in you can compare the micro-batching performance of Spark Vs. Storm Vs. Flink.

Additionally, decision should also consider

Highlighted

Re: spark vs flink vs storm

Expert Contributor

"storm operates on event basis" what does it mean ? can you please give example for the different events ? thank you.

Highlighted

Re: spark vs flink vs storm

Rising Star

Storm operates on 1 tuple / event/ message at a time; Spark operates on batches of messages. Event is whatever the message in your use case is. Events can represent log messages, messages in Kafka etc.

Highlighted

Re: spark vs flink vs storm

Rising Star

@heta desai please accept the answer if that helped address your question

Don't have an account?
Coming from Hortonworks? Activate your account here