Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Generating schema in FOREACH and then join - java.lang.ClassCastException: org.apache.pig.data.DataByteArray

avatar
Explorer

If I create a schema on load everything works fine. However I did not want to create whole schema for a few fields out of 100's. This script fails with

Unable to open iterator for alias A. Backend error : java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.math.BigInteger

H = LOAD '/user/horton/hands.txt' USING PigStorage(',');
FilteredHands = FILTER H BY (int)$10==6 AND (int)$5==6;
Hands = FOREACH H GENERATE $0 AS (HandID:biginteger), $5 AS (BigBet:double),  $10 AS (PlayersInHand:int); 
HP = LOAD '/user/horton/playerhands.txt' USING PigStorage(',');
HandPlayers = FOREACH HP GENERATE $1 AS (HandID:biginteger), $0 AS (PlayerName:chararray), $12 AS (AmtWon:double);
PlayersAndHands = JOIN Hands by HandID, HandPlayers by HandID;
A = FOREACH PlayersAndHands GENERATE  Hands.HandID;
DUMP A;

What am I doing wrong?

1 ACCEPTED SOLUTION

avatar
Master Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
4 REPLIES 4

avatar
Explorer

I cannot dump PlaeyersAndHands either. Both HandPlayers and Hands dump fine.

avatar
Master Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Explorer

Thank you so much. That worked. Is biginteger uncastable? When would I use bytearray vs. specific type? Interesting thing is if I LOAD with schema joins on biginteger works fine. Would you consider this a bug?

avatar
Master Mentor

can't tell without seeing your data. Can you try with schema join as well and confirm bytearray works in either condition.