Reply
Explorer
Posts: 15
Registered: ‎09-11-2014

Re: Reliable method for keeping counters in Spark

Does spark-streaming also have the double-counting problem for transformations on DStreams? I assume that under-the-hood they operate the same way, but I figured I'd better double-check.

Cloudera Employee
Posts: 481
Registered: ‎08-11-2014

Re: Reliable method for keeping counters in Spark

Yes it would be. The execution of transformations/actions is the same, just the source is different.