I am working on a aws dataset(email dataset -enron) . I just wanted to do a word count on all of the emails and find out the average. The files are zipped (Please see the screen shot attachment which shows how the actual data set looks like). Please if some one could help me by looking at the scrscreen-shot-2016-10-07-at-090457.pngeen shot that how I can do the word count processing using spark (scala preferably). I would really appreciate . Note: The actual datasize is 210 GB. I am planning to run an EMR cluster then perform the processing. ... View more

> Not sure can use select min(from_date) clause create table employees2_final stored as ORC tblproperties (‘orc.compress’=‘SNAPPY’) AS SELECT e.emp_no, e.birth_date,e.first_name, e.last_name, e.gender, select min(s.from_date) from new2_salaries s GROUP BY s.emp_no WHERE s.emp_no = e.emp_no as hire_date_new from employees e. ... View more

Please find the attached for the error screen grabscreen-shot-2016-04-27-at-082448.png Sqoop Query : sqoop import --driver com.mysql.jdbc.Driver --connect jdbc:mysql://sandbox.hortonworks.com/test --username root -P --m 1 --query "select empid, ename from emp WHERE '$CONDITIONS' AND eid > '200' " --target-dir /user/maria_dev ... View more