Hi, I have a small Voters list(name,gender,place,age) where I wanted to eliminate the voters whose age is <= 20.
I wanted to try streaming in pig. When I run the dump on stream its fails and is unable to idenetify python commands. I have attached python script, input data file, pig script and log file.
Could you guide where should I install the python in Sandbox. Thank you. Input: AAA,Female,Blr,40
BBB,Female,London,35
YYY,Female,Pondy,12
JJJ,Male,London,4
SSS,Female,Pondy,30 pig script in tez_local mode:
grunt> Voters = LOAD 'file:///user/revathy/pig/Voters.txt' USING PigStorage(',') AS (VoterName:chararray,Gender:chararray,Place:chararray,Age:int);
grunt> Eligible = STREAM Voters THROUGH `/root/revathy/pig/hello.py` AS (VoterName:chararray,Gender:chararray,Place:chararray,Age:int); Python script:(Tested in Python editor) import sys
THRESHOLD = 20 def filterVal(line,val4):
if int(val4) > THRESHOLD:
sys.stdout.writelines(line)
return
try:
for line in sys.stdin.readlines():
val1,val2,val3,val4 = str(line).split(",")
filterVal(line,val4)
except:
print "Error in try block" Log: /root/revathy/pig/hello.py:
line 1: import: command not found
/root/revathy/pig/hello.py:
line 2: THRESHOLD: command not found
/root/revathy/pig/hello.py:
line 3:
: command not found
... View more
