pig script from tutorial yield error when running on my sandbox

cid_peng — Fri, 29 Jul 2016 16:32:22 GMT

Hi, I was following the instructions in Tutorial 3 and use the same pig script in the tutorial.

The script:

a = LOAD 'geolocation' using org.apache.hive.hcatalog.pig.HCatLoader();
b = filter a by event != 'normal';
c = foreach b generate driverid, event, (int) '1' as occurance;
d = group c by driverid;
e = foreach d generate group as driverid, SUM(c.occurance) as t_occ;
g = LOAD 'drivermileage' using org.apache.hive.hcatalog.pig.HCatLoader();
h = join e by driverid, g by driverid;
final_data = foreach h generate $0 as driverid, $1 as events, $3 as totmiles, (float) $3/$1 as riskfactor;
store final_data into 'riskfactor' using org.apache.hive.hcatalog.pig.HCatStorer();

However, it gives me some errors like this:

WARNING: Use "yarn jar" to launch YARN applications.  
...
2016-07-29 09:11:18,610 [main] ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. Could not resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]  
Failed to parse: Pig script failed to parse:   
<file script.pig, line 1, column 29> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]  	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)  	
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1776)  	at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1484)  	at org.apache.pig.PigServer.parseAndBuild(PigServer.java:428)  	at org.apache.pig.PigServer.executeBatch(PigServer.java:453)  	at org.apache.pig.PigServer.executeBatch(PigServer.java:439)  	at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:171)  	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:234)  	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)  	at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)  	at org.apache.pig.Main.run(Main.java:502)  	at org.apache.pig.Main.main(Main.java:177)  	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)  	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  	at java.lang.reflect.Method.invoke(Method.java:606)  	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)  	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)  Caused by:   <file script.pig, line 1, column 29> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]  	at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1339)  	at org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1324)  	at org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5184)  	at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3515)  	at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)  	at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)  	at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)  	at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)  	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)  	... 17 more  Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]  	at org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:677)  	at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1336)  	... 25 more  2016-07-29 09:11:18,618 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]  Details at logfile: /hadoop/yarn/local/usercache/maria_dev/appcache/application_1469778965378_0006/container_e06_1469778965378_0006_01_000002/pig_1469783473914.log  2016-07-29 09:11:18,648 [main] INFO  org.apache.pig.Main - Pig script completed in 4 seconds and 883 milliseconds (4883 ms)

Re: pig script from tutorial yield error when running on my sandbox

lrouleau_ext — Fri, 29 Jul 2016 20:27:34 GMT

Hi,

I think you forgot to use the HCatalog parameter :

Try to use below command:

pig -useHCatalog -f yourscript.pig

Re: pig script from tutorial yield error when running on my sandbox

dfields — Tue, 30 Aug 2016 09:37:06 GMT

I had the same problem. The tutorial is ambiguous for someone who is new to Hadoop/Pig. A few more screenshits on how to use pig helper would be worth while. Also, .pig -useHCatalog -f yourscript.pig is never mentioned in the tutorial

question Re: pig script from tutorial yield error when running on my sandbox in Archives of Support Questions (Read Only)

pig script from tutorial yield error when running on my sandbox

Re: pig script from tutorial yield error when running on my sandbox

Re: pig script from tutorial yield error when running on my sandbox