I was successfully able to load the records into ‘/user/horton/flightdelays_clean’
While attempting the part 2 of TASK 03
I do the following:
A = LOAD '/user/horton/flightdelays_clean' USING PigStorage(',') ;
--- A has 29534 records
B = FILTER A BY $8 == 'DEN';
---[main] WARN org.apache.pig.newplan.BaseOperatorPlan - ---Encountered Warning IMPLICIT_CAST_TO_CHARARRAY 1 time(s)
Successfully stored 0 records in: "hdfs://namenode:8020/tmp/temp-577228131/tmp-2029278778"
There is no rows being imported into B. I thing the way I'm comparing $8 with DEN is not right.
I also tried to import A with Schema like the following:
grunt> A = LOAD '/user/horton/flightdelays_clean' USING PigStorage(',') AS (Year:int, Month:int, DayofMonth:int,DepTime:int, UniqueCarrier:chararray, FlightNum:int, ArrDelay:int, Origin:chararray, Dest:chararray);
2016-06-23 18:26:32,428 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - Encountered Warning IMPLICIT_CAST_TO_CHARARRAY 1 time(s).
B = FILTER A BY Dest == 'DEN';
I still end up wiht no rows being written to B;
I'm not facing errors but I see the string comparision is not working.