Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Generating schema in FOREACH and then join - java.lang.ClassCastException: org.apache.pig.data.DataByteArray

Explorer

If I create a schema on load everything works fine. However I did not want to create whole schema for a few fields out of 100's. This script fails with

Unable to open iterator for alias A. Backend error : java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.math.BigInteger

H = LOAD '/user/horton/hands.txt' USING PigStorage(',');
FilteredHands = FILTER H BY (int)$10==6 AND (int)$5==6;
Hands = FOREACH H GENERATE $0 AS (HandID:biginteger), $5 AS (BigBet:double),  $10 AS (PlayersInHand:int); 
HP = LOAD '/user/horton/playerhands.txt' USING PigStorage(',');
HandPlayers = FOREACH HP GENERATE $1 AS (HandID:biginteger), $0 AS (PlayerName:chararray), $12 AS (AmtWon:double);
PlayersAndHands = JOIN Hands by HandID, HandPlayers by HandID;
A = FOREACH PlayersAndHands GENERATE  Hands.HandID;
DUMP A;

What am I doing wrong?

1 ACCEPTED SOLUTION

Mentor

Replace biginteger datatype with bytearray.

View solution in original post

4 REPLIES 4

Explorer

I cannot dump PlaeyersAndHands either. Both HandPlayers and Hands dump fine.

Mentor

Replace biginteger datatype with bytearray.

Explorer

Thank you so much. That worked. Is biginteger uncastable? When would I use bytearray vs. specific type? Interesting thing is if I LOAD with schema joins on biginteger works fine. Would you consider this a bug?

Mentor

can't tell without seeing your data. Can you try with schema join as well and confirm bytearray works in either condition.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.