Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Reliable method for keeping counters in Spark

Solved Go to solution

Re: Reliable method for keeping counters in Spark

Explorer

Does spark-streaming also have the double-counting problem for transformations on DStreams? I assume that under-the-hood they operate the same way, but I figured I'd better double-check.

Re: Reliable method for keeping counters in Spark

Master Collaborator

Yes it would be. The execution of transformations/actions is the same, just the source is different.