Support Questions
Find answers, ask questions, and share your expertise

Unable to cast output type for "Over" operator.

Unable to cast output type for "Over" operator.

Expert Contributor


I am unable to typecast the column from chararray to int for the column output from over operator. I am using over operator to derive the value of a column from next record. I define the over operator like this.

DEFINE Over org.apache.pig.piggybank.evaluation.Over('chararray');

So that output is chararray. Now the following code produces chararray, I could see the type as chararray using describe command. But unable to cast it to int

dist_dates = foreach expr_d_set {
distinct_dates = distinct LND_DATA_REQD.file_date;
sorted_ip = order distinct_dates by file_date;
stitched = Stitch(sorted_ip, Over(sorted_ip.file_date, 'lead',0,1,1,'99991231'));
generate flatten(group) as (queueid:chararray, acdid:int), flatten(stitched) as (file_date:int, expr_d:chararray) ;
grunt> describe dist_dates;
dist_dates: {queueid: chararray,acdid: int,file_date: int,expr_d: chararray} --> Notice the chararray here?
I  = FOREACH dist_dates generate (int)expr_d;

I try this to cast to integer. But failing with error message "java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String" - I dont understand why this cast is not working, I am able to cast other columns to integer though using the same syntax. But I am able to convert the same to Date.. Can someone please suggest what I am missing here?

I  = FOREACH dist_dates generate ToDate(expr_d,'YYYYMMdd') as EXPR_D;