I am new to oozie and can anyone explain what is the purpouse of input-events and output-events.I have gone through the manual but still it is not clear.can anyone explain input-event in below case. <coordinator-app name="MY_APP" frequency="1440" start="2009-02-01T00:00Z" end="2009-02-07T00:00Z" timezone="UTC" xmlns="uri:oozie:coordinator:0.1"> <datasets> <dataset name="input1" frequency="60" initial-instance="2009-01-01T00:00Z" timezone="UTC"> <uri-template>hdfs://localhost:9000/tmp/revenue_feed/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template> </dataset> </datasets> <input-events> <data-in name="coordInput1" dataset="input1"> <start-instance>${coord:current(-23)}</start-instance> <end-instance>${coord:current(0)}</end-instance> </data-in> </input-events> <action> <workflow> <app-path>hdfs://localhost:9000/tmp/workflows</app-path> </workflow> </action> </coordinator-app> ... View more

I have grouped data as shown below: group_data = GROUP student_details by age; dump group_data; (21,{(4,Preethi,Agarwal,21,9848022330,Pune),(1,Rajiv,Reddy,21,9848022337,Hydera bad)}) (22,{(3,Rajesh,Khanna,22,9848022339,Delhi),(2,siddarth,Battacharya,22,984802233 8,Kolkata)}) (23,{(6,Archana,Mishra,23,9848022335,Chennai),(5,Trupthi,Mohanthy,23,9848022336 ,Bhuwaneshwar)}) (24,{(8,Bharathi,Nambiayar,24,9848022333,Chennai),(7,Komal,Nayak,24,9848022334, trivendram)}) Required output from group_data is as below: (23,{(6,Archana,Mishra,23,9848022335,Chennai)}) (24,{(8,Bharathi,Nambiayar,24,9848022333,Chennai)}) Tried below thing: First Try:- t = foreach group_data generate group, FLATTEN(student_details1); filter_data = FILTER t BY student_details::city == 'Chennai'; (23,6,Archana,Mishra,23,9848022335,Chennai) (24,8,Bharathi,Nambiayar,24,9848022333,Chennai) Second Try:- t_1 = FOREACH group_data { t_2 = FILTER student_details BY city == 'Chennai'; GENERATE group,t_2 AS failTime; }; (21,{}) (22,{}) (23,{(6,Archana,Mishra,23,9848022335,Chennai)}) (24,{(8,Bharathi,Nambiayar,24,9848022333,Chennai)}) ... View more

x = LOAD '/pigdata/source.txt' using PigStorage(',') As (exchange:chararray, symbol:chararray, date:chararray, open:double, high:double, low:double, close:double, volume:long, adj_close:double); y = GROUP x by symbol; z2 = foreach y generate x.exchange as exchange1; dump z2; ({(NASDAQ),(NASDAQ),(NASDAQ),(ICICI),(ICICI),(ICICI),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ)}) ({(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ)}) z4 = distinct z2; dump z4; ({(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ)}) ({(NASDAQ),(NASDAQ),(NASDAQ),(ICICI),(ICICI),(ICICI),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ),(NASDAQ)}) clarification:- How distinct will work with bags?For tuples it is clear and what will happen if i am using distinct with bags?dump z4 is not clear to me. ... View more

Below is my source in HDFS: /abc/ Hadoop is an open source MR is to process data in hadoop. Hadoop has a good eco system. I want to do below opearation filter_records = FILTER ya BY $0 MATCHES '.*Hadoop.*'; but load command is unsuccessful.Could anybody provide input on load statement? grunt> ya = load '/abc/' USING TextLoader(); 2016-11-17 21:00:14,470 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - Encountered Warning IMPLICIT_CAST_TO_CHARARRAY 1 time(s). grunt> yab = load '/abc/'; 2016-11-17 21:00:50,199 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - Encountered Warning IMPLICIT_CAST_TO_CHARARRAY 1 time(s). grunt> ... View more