question Re: Pig Accumulator in Spark in Archives of Support Questions (Read Only) https://community.cloudera.com/t5/Archives-of-Support-Questions/Pig-Accumulator-in-Spark/m-p/130819#M31363 <P>I'm not aware of the concept of Spark's Accumulators exposed as "first-class" objects in Pig and have always advised that you would need to build a UDF for such activities if you couldn't simply get away with filtering the things to count (such as "good" records and "rejects") into separate aliases then count them up. </P><P>Here is a blog post going down the UDF path; <A href="https://dzone.com/articles/counters-apache-pig" target="_blank">https://dzone.com/articles/counters-apache-pig</A>. </P><P>Good luck &amp; I'd love to hear if there was something I've been missing all along directly from Pig.</P> Fri, 10 Jun 2016 21:41:22 GMT LesterMartin 2016-06-10T21:41:22Z Pig Accumulator in Spark https://community.cloudera.com/t5/Archives-of-Support-Questions/Pig-Accumulator-in-Spark/m-p/130818#M31362 <P>are there functions out there that utilize something like the accumulator interface in Pig where the data doesn't have to stay in memory?</P> Thu, 09 Jun 2016 22:26:09 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Pig-Accumulator-in-Spark/m-p/130818#M31362 TimothySpann 2016-06-09T22:26:09Z Re: Pig Accumulator in Spark https://community.cloudera.com/t5/Archives-of-Support-Questions/Pig-Accumulator-in-Spark/m-p/130819#M31363 <P>I'm not aware of the concept of Spark's Accumulators exposed as "first-class" objects in Pig and have always advised that you would need to build a UDF for such activities if you couldn't simply get away with filtering the things to count (such as "good" records and "rejects") into separate aliases then count them up. </P><P>Here is a blog post going down the UDF path; <A href="https://dzone.com/articles/counters-apache-pig" target="_blank">https://dzone.com/articles/counters-apache-pig</A>. </P><P>Good luck &amp; I'd love to hear if there was something I've been missing all along directly from Pig.</P> Fri, 10 Jun 2016 21:41:22 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Pig-Accumulator-in-Spark/m-p/130819#M31363 LesterMartin 2016-06-10T21:41:22Z