Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unable to cast output type for "Over" operator.


Unable to cast output type for "Over" operator.

Expert Contributor


I am unable to typecast the column from chararray to int for the column output from over operator. I am using over operator to derive the value of a column from next record. I define the over operator like this.

DEFINE Over org.apache.pig.piggybank.evaluation.Over('chararray');

So that output is chararray. Now the following code produces chararray, I could see the type as chararray using describe command. But unable to cast it to int

dist_dates = foreach expr_d_set {
distinct_dates = distinct LND_DATA_REQD.file_date;
sorted_ip = order distinct_dates by file_date;
stitched = Stitch(sorted_ip, Over(sorted_ip.file_date, 'lead',0,1,1,'99991231'));
generate flatten(group) as (queueid:chararray, acdid:int), flatten(stitched) as (file_date:int, expr_d:chararray) ;
grunt> describe dist_dates;
dist_dates: {queueid: chararray,acdid: int,file_date: int,expr_d: chararray} --> Notice the chararray here?
I  = FOREACH dist_dates generate (int)expr_d;

I try this to cast to integer. But failing with error message "java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String" - I dont understand why this cast is not working, I am able to cast other columns to integer though using the same syntax. But I am able to convert the same to Date.. Can someone please suggest what I am missing here?

I  = FOREACH dist_dates generate ToDate(expr_d,'YYYYMMdd') as EXPR_D;