Support Questions

Find answers, ask questions, and share your expertise

Generating schema in FOREACH and then join - java.lang.ClassCastException: org.apache.pig.data.DataByteArray

avatar
Explorer

If I create a schema on load everything works fine. However I did not want to create whole schema for a few fields out of 100's. This script fails with

Unable to open iterator for alias A. Backend error : java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.math.BigInteger

H = LOAD '/user/horton/hands.txt' USING PigStorage(',');
FilteredHands = FILTER H BY (int)$10==6 AND (int)$5==6;
Hands = FOREACH H GENERATE $0 AS (HandID:biginteger), $5 AS (BigBet:double),  $10 AS (PlayersInHand:int); 
HP = LOAD '/user/horton/playerhands.txt' USING PigStorage(',');
HandPlayers = FOREACH HP GENERATE $1 AS (HandID:biginteger), $0 AS (PlayerName:chararray), $12 AS (AmtWon:double);
PlayersAndHands = JOIN Hands by HandID, HandPlayers by HandID;
A = FOREACH PlayersAndHands GENERATE  Hands.HandID;
DUMP A;

What am I doing wrong?

1 ACCEPTED SOLUTION

avatar
Master Mentor

Replace biginteger datatype with bytearray.

View solution in original post

4 REPLIES 4

avatar
Explorer

I cannot dump PlaeyersAndHands either. Both HandPlayers and Hands dump fine.

avatar
Master Mentor

Replace biginteger datatype with bytearray.

avatar
Explorer

Thank you so much. That worked. Is biginteger uncastable? When would I use bytearray vs. specific type? Interesting thing is if I LOAD with schema joins on biginteger works fine. Would you consider this a bug?

avatar
Master Mentor

can't tell without seeing your data. Can you try with schema join as well and confirm bytearray works in either condition.