question Re: Pig on Tez causes unable to store alias in Archives of Support Questions (Read Only)

Pig on Tez causes unable to store alias

petri_koski — Fri, 20 May 2016 21:00:49 GMT

Hello,

I have Tez 0.7.0 and HDP 2.4.0.0.169. I use pig latin quite a lot in Mapreduce -mode. Today I tried Tez mode (From my java code .. ) everything seems to work fine, but in same pig latin script in Mapreduce stores alias Ok, but in tez -mode I get "ERROR 1002: Unable to store alias secondorder". I checked the script and nothing wrong there. Here is my script: (When I check path tebs/results/Ravintolamyynnintop20 from HDFS, its there stored ok .. But still I get this error Unable to store alias secondorder and show stops .. ) what could be wrong ? Some version conflict with my Java code vs. HDP ?

splittedII = load 'tebs/data/currentmon*.*' using PigStorage(';') as (id:chararray,fu:chararray,fa:chararray,myynti:chararray);

splittedI = FILTER splittedII BY NOT($3 MATCHES '.*Ko.*') AND NOT($2 MATCHES '.*Yht.*') AND NOT($1 MATCHES '.*Yht.*');




onlyrestaurants = FILTER splittedI BY ($1 MATCHES '.*23021 RUOKA.*') OR ($1 MATCHES '.*23022 SUOLAINEN.*') OR ($1 MATCHES '.*23023 MAKEA.*') OR ($1 MATCHES '.*23024 PIKARUOKA.$




partly = foreach onlyrestaurants generate $1,$2,REPLACE(myynti, ',','.');

store partly into 'teb/tempten';

partly = load 'teb/tempten' using PigStorage('\t') as (id:chararray,fu:chararray,myynti:double);




grpded = group partly by ($0,$1);

summed = foreach grpded generate FLATTEN(group) AS (id,fu),SUM(partly.$2);

ordered = order summed by $2 DESC;

grp = group ordered by $0;

top10 = foreach grp {

sorted = order ordered by $2 desc;

top = limit sorted 20;

total = SUM(ordered.$2);

generate group,FLATTEN(top),FLATTEN(total);

};

secondorder = ORDER top10 by $4 DESC;

store secondorder into 'tebs/results/Ravintolamyynnintop20';

Re: Pig on Tez causes unable to store alias

pradeep_bhadani — Fri, 20 May 2016 23:10:14 GMT

@petri koskican you share Pig job Logs?

Re: Pig on Tez causes unable to store alias

petri_koski — Sat, 21 May 2016 03:25:12 GMT

Hello! Thanks for the quick reply. Here comes log file for this job: http://pastebin.ca/3605448 Don't mind about the lines after "starting make visualisation" .. They fail for reason in this case. I am stunned, because that script stores one time temp -file, but when it comes to second time .. unable to store .. I hope you can help me with this one. @Pradeep Bhadani

Re: Pig on Tez causes unable to store alias

petri_koski — Sun, 22 May 2016 17:21:06 GMT

I found solution: Problem was about versions: I was using Tez 0.7.0 from Maven central, and hadoop 2.7.1 .. When I used 0.7.1 version from Tez this STORE -problem is gone. So problem solved.