Hello,
I have Tez 0.7.0 and HDP 2.4.0.0.169. I use pig latin quite a lot in Mapreduce -mode. Today I tried Tez mode (From my java code .. ) everything seems to work fine, but in same pig latin script in Mapreduce stores alias Ok, but in tez -mode I get "ERROR 1002: Unable to store alias secondorder". I checked the script and nothing wrong there. Here is my script: (When I check path tebs/results/Ravintolamyynnintop20 from HDFS, its there stored ok .. But still I get this error Unable to store alias secondorder and show stops .. ) what could be wrong ? Some version conflict with my Java code vs. HDP ?
splittedII = load 'tebs/data/currentmon*.*' using PigStorage(';') as (id:chararray,fu:chararray,fa:chararray,myynti:chararray);
splittedI = FILTER splittedII BY NOT($3 MATCHES '.*Ko.*') AND NOT($2 MATCHES '.*Yht.*') AND NOT($1 MATCHES '.*Yht.*');
onlyrestaurants = FILTER splittedI BY ($1 MATCHES '.*23021 RUOKA.*') OR ($1 MATCHES '.*23022 SUOLAINEN.*') OR ($1 MATCHES '.*23023 MAKEA.*') OR ($1 MATCHES '.*23024 PIKARUOKA.$
partly = foreach onlyrestaurants generate $1,$2,REPLACE(myynti, ',','.');
store partly into 'teb/tempten';
partly = load 'teb/tempten' using PigStorage('\t') as (id:chararray,fu:chararray,myynti:double);
grpded = group partly by ($0,$1);
summed = foreach grpded generate FLATTEN(group) AS (id,fu),SUM(partly.$2);
ordered = order summed by $2 DESC;
grp = group ordered by $0;
top10 = foreach grp {
sorted = order ordered by $2 desc;
top = limit sorted 20;
total = SUM(ordered.$2);
generate group,FLATTEN(top),FLATTEN(total);
};
secondorder = ORDER top10 by $4 DESC;
store secondorder into 'tebs/results/Ravintolamyynnintop20';
