Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Passing parameters to pig script from file issue

avatar
Rising Star

pig -x mapreduce /home/hadoop/test.pig -param_file=hdfs://ip-XXX-XX-XX-XXX.ec2.internal:8020/home/hadoop/adh_time

I am trying to run the above command and its not working saying undefined parameter i tired multiple ways and nothing works.Any help would be appreciated.

Value inside the adh_time is input = 2015-06-16T14:59:55.000Z

Error:

16/01/26 20:56:38 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL 16/01/26 20:56:38 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE 16/01/26 20:56:38 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType 46 [main] INFO org.apache.pig.Main - Apache Pig version 0.14.0-amzn-0 (r: unknown) compiled Nov 16 2015, 21:51:33 16/01/26 20:56:38 INFO pig.Main: Apache Pig version 0.14.0-amzn-0 (r: unknown) compiled Nov 16 2015, 21:51:33 47 [main] INFO org.apache.pig.Main - Logging error messages to: /mnt/var/log/pig/pig_1453841798872.log 16/01/26 20:56:38 INFO pig.Main: Logging error messages to: /mnt/var/log/pig/pig_1453841798872.log 789 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/hadoop/.pigbootup not found 16/01/26 20:56:39 INFO util.Utils: Default bootup file /home/hadoop/.pigbootup not found 810 [main] ERROR org.apache.pig.impl.PigContext - Undefined parameter : input 16/01/26 20:56:39 ERROR impl.PigContext: Undefined parameter : input 828 [main] ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. org.apache.pig.tools.parameters.ParameterSubstitutionException: Undefined parameter : input 16/01/26 20:56:39 ERROR pig.Main: ERROR 2997: Encountered IOException. org.apache.pig.tools.parameters.ParameterSubstitutionException: Undefined parameter : input Details at logfile: /mnt/var/log/pig/pig_1453841798872.log 848 [main] INFO org.apache.pig.Main - Pig script completed in 952 milliseconds (952 ms) 16/01/26 20:56:39 INFO pig.Main: Pig script completed in 952 milliseconds (952 ms)

1 ACCEPTED SOLUTION

avatar
Expert Contributor

I guess it is not considering param file

try this,

pig -param_file=hdfs://ip-XXX-XX-XX-XXX.ec2.internal:8020/home/hadoop/adh_time /home/hadoop/test.pig

writing param_file at the end encountering same issue for me too

View solution in original post

6 REPLIES 6

avatar
Super Collaborator

Please paste the contents of /mnt/var/log/pig/pig_1453841798872.log

avatar
Expert Contributor

I guess it is not considering param file

try this,

pig -param_file=hdfs://ip-XXX-XX-XX-XXX.ec2.internal:8020/home/hadoop/adh_time /home/hadoop/test.pig

writing param_file at the end encountering same issue for me too

avatar
Rising Star

I tried both in my home driectory and it worked fine and also it worked fine when i had my parm file on s3. But its not working when i try to do it from hdfs.

avatar
Master Mentor

@Pradeep Allu did you try Suresh's idea? Can you accept the answer or add more detail

avatar
Expert Contributor

@Pradeep Allu

If your parameter parameter file is in hdfs then make sure your pig script should also be in hdfs not in local file system.

Try placing your pig script also in HDFS .

avatar
Master Mentor

@Pradeep Allu try to pass just the directory path rather than full path including hdfs. Maybe put the path in quotes also?