Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: The Cloudera Community will undergo maintenance on Saturday, August 17 at 12:00am PDT. See more info here.

Reliable method for keeping counters in Spark

SOLVED Go to solution

Re: Reliable method for keeping counters in Spark

Explorer

Does spark-streaming also have the double-counting problem for transformations on DStreams? I assume that under-the-hood they operate the same way, but I figured I'd better double-check.

Highlighted

Re: Reliable method for keeping counters in Spark

Master Collaborator

Yes it would be. The execution of transformations/actions is the same, just the source is different.