Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Pig Error: Calling the Attributes

Pig Error: Calling the Attributes

Hi,

I am Using the below code to perform the Order Operation but it throwing the error that "Invalid field projection" where As the The Relation Has the column.

grunt> byts = ORDER B BY JB_DLT::job_id DESC; 

ERROR

2016-09-20 07:32:56,815 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1025: <line 99, column 18> Invalid field projection. Projected field [JB_DLT::job_id] does not exist in schema: group:tuple(JB_ASGNMNT::JB_DLT::job_id:bigdecimal,JOB_ASNMNT_STS_DLT::job_assignment_status_cd:chararray),a:bag{:tuple(JB_ASGNMNT::JB_DLT::job_id:bigdecimal,JOB_ASNMNT_STS_DLT::job_assignment_status_cd:chararray,JOB_ASNMNT_STS_DLT::job_assignment_status_id:bigdecimal,JOB_ASNMNT_STS_DLT::actual_status_dt:datetime)}.

grunt> describe B; 

B: {group: (JB_ASGNMNT::JB_DLT::job_id: bigdecimal,JOB_ASNMNT_STS_DLT::job_assignment_status_cd: chararray),a: {(JB_ASGNMNT::JB_DLT::job_id: bigdecimal,JOB_ASNMNT_STS_DLT::job_assignment_status_cd: chararray,JOB_ASNMNT_STS_DLT::job_assignment_status_id: bigdecimal,JOB_ASNMNT_STS_DLT::actual_status_dt: datetime)}}

How to read this column data in the ORDER BY Clause. Please share your thoughts..

========================= Here is The Entire Code and Sample Data =====================

J_B = LOAD 'ABCD' USING org.apache.hive.hcatalog.pig.HCatLoader(); 
J_B_SQ = FOREACH J_B GENERATE wo_id,jb_id,ls_upd_ts; 
JB_DLT = FILTER J_B_SQ BY ls_upd_ts >= SubtractDuration(CurrentTime(),'P1D'); 
J_A_TBL = LOAD 'EFGH' USING org.apache.hive.hcatalog.pig.HCatLoader(); 
J_A_SQ = FOREACH J_A_TBL GENERATE jb_id,J_A_ID,ToString(eff_ts,'YYYY-MM-dd') AS DLT_CLMN; 
J_A_DLT = FILTER J_A_SQ  BY DLT_CLMN == '9999-12-31'; 
J_A_STS_TBL = LOAD 'IJKL' USING org.apache.hive.hcatalog.pig.HCatLoader(); 
J_A_STS_SQ = FOREACH J_A_STS_TBL GENERATE J_A_ID,j_ast_sts_id,J_A_S_CD,wk_cd,e_st_ts,ToString(eff_ts,'YYYY-MM-dd') AS DLT_CLMN; 
J_A_STS_DLT = FILTER J_A_STS_SQ  BY DLT_CLMN == '9999-12-31'; 
J_ASGN =  JOIN JB_DLT by job_id, J_A_DLT by job_id;
J_ASGN_STS = JOIN J_ASGN by J_A_DLT::J_A_ID,J_A_STS_DLT by J_A_ID;
a = FOREACH J_ASGN_STS GENERATE JB_DLT::job_id,(J_A_STS_DLT::J_A_S_CD is null ? 'XXXX' : J_A_STS_DLT::J_A_S_CD) AS J_A_S_CD,J_A_STS_DLT::j_a_s_id, J_A_STS_DLT::actual_status_dt;
B = GROUP a BY (JB_DLT::job_id,J_A_STS_DLT::J_A_S_CD); 
C = FOREACH B {byts = ORDER B BY JB_DLT::job_id DESC;
                newest = LIMIT byts 1;
                GENERATE FLATTEN(newest);
               };

Sample Data:

jb_id J_A_ID J_A_S_CD

123 421 DIS

123 421 REJ

123 421T EN

536 386 ACC

536 386 COM

536 386 DIS

OUT PUT Will be the Same

1 REPLY 1

Re: Pig Error: Calling the Attributes

Guru

Not too clear from the code you have provided what you are trying to order. It is clear that you have grouped your data. If you want to order these grouped tuples by one of its member fields, see:

Don't have an account?
Coming from Hortonworks? Activate your account here