Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

spark streaming vs apache Flink ???

Highlighted

spark streaming vs apache Flink ???

Contributor

whats the actual difference between the two in terms of performing transformations on the live data coming in and also with data thats already in just few mins ago i mean combining live streaming + sliding window processing?? i.e. combining the data which just arrived few mins ago with the data that is coming in live..

3 REPLIES 3

Re: spark streaming vs apache Flink ???

Mentor
@surender nath reddy kudumula

Flink is a pure streaming framework with a lot of windowing capabilities. Spark streaming is still new and just coming from Spark Summit, Databricks is investing in Spark Streaming heavily going forward. Spark Streaming is a micro-batch operation. Flink is not covered by our support and Spark Streaming is, consider that when you make a decision on the framework. At the Summit, Databricks had mentioned that time based aggregations will be a focus for Spark Streaming in the next few releases. Spark Summit website will have slides posted within weeks.

Highlighted

Re: spark streaming vs apache Flink ???

Contributor

@Artem Ervits thanks for the reply How about storm can we use time based aggregations using storm??? So with existing spark streaming api in 1.5 as its not experimental and 1.6 is experimental can we perform time based aggregations using statefull RDDs, window based transformations in spark streaming??

Highlighted

Re: spark streaming vs apache Flink ???

@vshukla @surender nath reddy kudumula

I have tagged Vinay from Spark team to get you better understanding

Don't have an account?
Coming from Hortonworks? Activate your account here