Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

GENERATE AVG coming in a tuple fromat after grouping PIG

Highlighted

GENERATE AVG coming in a tuple fromat after grouping PIG

Rising Star

Bonjour everyone,

Listen guys, I have previously grouped a relation and the output was declared as a new scheme for a new relation. Then I create a second grouping in the output schema to apply an AVG function bu, guys seems like this new scheme is very different and I'm not able to reach out the required column to apply the AVG.

Details:

First grouping on a relation called trips_dp to get a new output relation with independent schema

trips_gr_dp = GROUP trips_dp BY (departure_station,day,hour);
trips_dp_qty = FOREACH trips_gr_dp { 
 qty_trips = COUNT(trips_dp); 
 GENERATE FLATTEN(group) as (departure_station,day,hour), qty_trips as (qty_dp_trip:int);
 };

Now the output and schema status for the new relation is (image 1):

Then, finally I need to get the AVG on hour for whole days within this relation so I grouped and apply the required transformation:

 trips_dp_qty_gr = GROUP trips_dp_qty by (departure_station,hour);
  
  trips_dp_myn = FOREACH trips_dp_qty_gr {
              myn_trips = AVG(trips_dp_qty.qty_dp_trip);
              GENERATE FLATTEN(group) as (departure_station,hour), myn_trips as (myn_trips:float);
              };

My Issue: As you can see in the second group I try to get an AVG from the previous relation trips_dp_qty in the column qty_dp_trip. Unfortunately, I got an error when I run the script because seems like pig can't reach out the desired schema.

40631-2017-10-03-19-06-30.png


1.png