here is my deal today. Well, I have created a relation as result of a couple of transformations after have read the relation from hive. the thing is that I want to store the final relation after a couple of analysis back in Hive but I can't. Let see that in my code much clear. The first String is when I LOAD from Hive and transform my result: july = LOAD 'POC.july' USING org.apache.hive.hcatalog.pig.HCatLoader ; july_cl = FOREACH july GENERATE GetDay(ToDate(start_date)) as day:int,start_station,duration; jul_cl_fl = FILTER july_cl BY day==31; july_gr = GROUP jul_cl_fl BY (day,start_station); july_result = FOREACH july_gr { total_dura = SUM(jul_cl_fl.duration); avg_dura = AVG(jul_cl_fl.duration); qty_trips = COUNT(jul_cl_fl); GENERATE FLATTEN(group),total_dura,avg_dura,qty_trips; }; So, now when I try to store the relation july_result I can't because the schema has changed and I suppose that it's not compatible with Hive: STORE july_result INTO 'poc.july_analysis' USING org.apache.hive.hcatalog.pig.HCatStorer (); Even if I have tried to set a special scheme for the final relation I haven't figured it out. july_result = FOREACH july_gr { total_dura = SUM(jul_cl_fl.duration); avg_dura = AVG(jul_cl_fl.duration); qty_trips = COUNT(jul_cl_fl); GENERATE FLATTEN(group) as (day:int),total_dura as (total_dura:int),avg_dura as (avg_dura:int),qty_trips as (qty_trips:int); }; PDS: the table in hive exists previusly!! ... View more

Hey everyone, My case today is a little weird for me cause according to me I'm running the right scripts but well anyways, the thing is that I need load a file with a char structure but then I need to delete the headers and set the right structure relation. Here my code: Finally, I'm trying to perform a sum using this new structure but Pig always according with the log return an error casting the column with the new format as long for salary and int for the year so is like Pig won't be able to get the new structure. Error: Could some of our gurus let me know the right way to get a nice transformation and perform my scripts. thanks so much ... View more

Hi guys, I have been struggling with my Pig code and I haven't arrived at desired result so this is why I'm knocking you guys. Well, I have a file with some information and my idea is to get a counting by reference number. so as an overview I have done: So the third step worked but the problem is that it generates a huge tuple to include the reference number by each tuple in my grouping bag that contains the number so the output it's like: Then I tried the fourth step but although I got the counting list I missed the reference_number so I would like to get the same list but just once the reference code. Thanks so much for your help team. @Lester Martin ... View more