Created 09-26-2016 08:34 AM
If I create a schema on load everything works fine. However I did not want to create whole schema for a few fields out of 100's. This script fails with
Unable to open iterator for alias A. Backend error : java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.math.BigInteger
H = LOAD '/user/horton/hands.txt' USING PigStorage(','); FilteredHands = FILTER H BY (int)$10==6 AND (int)$5==6; Hands = FOREACH H GENERATE $0 AS (HandID:biginteger), $5 AS (BigBet:double), $10 AS (PlayersInHand:int); HP = LOAD '/user/horton/playerhands.txt' USING PigStorage(','); HandPlayers = FOREACH HP GENERATE $1 AS (HandID:biginteger), $0 AS (PlayerName:chararray), $12 AS (AmtWon:double); PlayersAndHands = JOIN Hands by HandID, HandPlayers by HandID; A = FOREACH PlayersAndHands GENERATE Hands.HandID; DUMP A;
What am I doing wrong?
Created 09-26-2016 09:43 AM
Replace biginteger datatype with bytearray.
Created 09-26-2016 09:29 AM
I cannot dump PlaeyersAndHands either. Both HandPlayers and Hands dump fine.
Created 09-26-2016 09:43 AM
Replace biginteger datatype with bytearray.
Created 09-26-2016 09:51 AM
Thank you so much. That worked. Is biginteger uncastable? When would I use bytearray vs. specific type? Interesting thing is if I LOAD with schema joins on biginteger works fine. Would you consider this a bug?
Created 09-26-2016 12:36 PM
can't tell without seeing your data. Can you try with schema join as well and confirm bytearray works in either condition.