Created 09-26-2016 08:34 AM
If I create a schema on load everything works fine. However I did not want to create whole schema for a few fields out of 100's. This script fails with
Unable to open iterator for alias A. Backend error : java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.math.BigInteger
H = LOAD '/user/horton/hands.txt' USING PigStorage(',');
FilteredHands = FILTER H BY (int)$10==6 AND (int)$5==6;
Hands = FOREACH H GENERATE $0 AS (HandID:biginteger), $5 AS (BigBet:double), $10 AS (PlayersInHand:int);
HP = LOAD '/user/horton/playerhands.txt' USING PigStorage(',');
HandPlayers = FOREACH HP GENERATE $1 AS (HandID:biginteger), $0 AS (PlayerName:chararray), $12 AS (AmtWon:double);
PlayersAndHands = JOIN Hands by HandID, HandPlayers by HandID;
A = FOREACH PlayersAndHands GENERATE Hands.HandID;
DUMP A;What am I doing wrong?
Created 09-26-2016 09:43 AM
Replace biginteger datatype with bytearray.
Created 09-26-2016 09:29 AM
I cannot dump PlaeyersAndHands either. Both HandPlayers and Hands dump fine.
Created 09-26-2016 09:43 AM
Replace biginteger datatype with bytearray.
Created 09-26-2016 09:51 AM
Thank you so much. That worked. Is biginteger uncastable? When would I use bytearray vs. specific type? Interesting thing is if I LOAD with schema joins on biginteger works fine. Would you consider this a bug?
Created 09-26-2016 12:36 PM
can't tell without seeing your data. Can you try with schema join as well and confirm bytearray works in either condition.