https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-process-a-word-count-on-zipped-files-in-spark/m-p/118953#M42965 <P>I am working on a aws dataset(email dataset -enron) . I just wanted to do a word count on all of the emails and find out the average. The files are zipped (Please see the screen shot attachment which shows how the actual data set looks like). Please if some one could help me by looking at the scr<A href="https://community.cloudera.com/legacyfs/online/attachments/8308-screen-shot-2016-10-07-at-090457.png">screen-shot-2016-10-07-at-090457.png</A>een shot that how I can do the word count processing using spark (scala preferably). I would really appreciate . </P><P>Note: The actual datasize is 210 GB. I am planning to run an EMR cluster then perform the processing. </P> Fri, 07 Oct 2016 15:10:45 GMT sanjeevanmahaja 2016-10-07T15:10:45Z https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-process-a-word-count-on-zipped-files-in-spark/m-p/118953#M42965 <P>I am working on a aws dataset(email dataset -enron) . I just wanted to do a word count on all of the emails and find out the average. The files are zipped (Please see the screen shot attachment which shows how the actual data set looks like). Please if some one could help me by looking at the scr<A href="https://community.cloudera.com/legacyfs/online/attachments/8308-screen-shot-2016-10-07-at-090457.png">screen-shot-2016-10-07-at-090457.png</A>een shot that how I can do the word count processing using spark (scala preferably). I would really appreciate . </P><P>Note: The actual datasize is 210 GB. I am planning to run an EMR cluster then perform the processing. </P> Fri, 07 Oct 2016 15:10:45 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-process-a-word-count-on-zipped-files-in-spark/m-p/118953#M42965 sanjeevanmahaja 2016-10-07T15:10:45Z https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-process-a-word-count-on-zipped-files-in-spark/m-p/118954#M42966 <P>These should be helpful</P><P><A href="http://stackoverflow.com/questions/32080475/how-to-read-a-zip-containing-multiple-files-in-apache-spark" target="_blank">http://stackoverflow.com/questions/32080475/how-to-read-a-zip-containing-multiple-files-in-apache-spark</A></P><P><A href="http://stackoverflow.com/questions/28569788/how-to-open-stream-zip-files-through-spark" target="_blank">http://stackoverflow.com/questions/28569788/how-to-open-stream-zip-files-through-spark</A></P> Fri, 07 Oct 2016 20:19:54 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-process-a-word-count-on-zipped-files-in-spark/m-p/118954#M42966 gkeys 2016-10-07T20:19:54Z