HDPCD Practice Exam TASK 03

[#1]

Hi,

I'm attempting the TASK 03 on Practice test.

PART 1 I was successfully able to load the records into ‘/user/horton/flightdelays_clean’

While attempting the part 2 of TASK 03 I do the following:

A = LOAD '/user/horton/flightdelays_clean' USING PigStorage(',') ;

--- A has 29534 records B = FILTER A BY $8 == 'DEN'; ---[main] WARN org.apache.pig.newplan.BaseOperatorPlan - ---Encountered Warning IMPLICIT_CAST_TO_CHARARRAY 1 time(s) Output(s): Successfully stored 0 records in: "hdfs://namenode:8020/tmp/temp-577228131/tmp-2029278778"

There is no rows being imported into B. I thing the way I'm comparing $8 with DEN is not right.

I also tried to import A with Schema like the following: grunt> A = LOAD '/user/horton/flightdelays_clean' USING PigStorage(',') AS (Year:int, Month:int, DayofMonth:int,DepTime:int, UniqueCarrier:chararray, FlightNum:int, ArrDelay:int, Origin:chararray, Dest:chararray); 2016-06-23 18:26:32,428 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - Encountered Warning IMPLICIT_CAST_TO_CHARARRAY 1 time(s).

B = FILTER A BY Dest == 'DEN';

I still end up wiht no rows being written to B;

I'm not facing errors but I see the string comparision is not working.

How can i fix this?

@rich

[#2 · Accepted solution]

Using a '\t' delimiter instead of a ',' works . This is resolved . Thanks.

[#3]

Just a couple of helpful tips:

1. The tab is the default delimiter, so you can just use LOAD '/user/horton/flightdelays_clean'
2. The answers to the Practice Exam tasks are on the Ubuntu client in the /home/horton/solutions folder.