Hi,
Hoping someone can advise. I am running a few lines of Pig code where I get an error.
When I run the four lines of code below in Pig it works fine:
allairports = LOAD 'default.airport' USING org.apache.hive.hcatalog.pig.HCatLoader();
gbairports = filter allairports by iso_country == 'GB';
gbcols = foreach gbairports generate $13, $3, $6;
dump gbcols;
However, when I replace the last line (as below) with a store statement, I get an error.
allairports = LOAD 'default.airport' USING org.apache.hive.hcatalog.pig.HCatLoader();
gbairports = filter allairports by iso_country == 'GB';
gbcols = foreach gbairports generate $13, $3, $6;
store gbcols into '/user/hue/tutorials' using pigstorage(',');
The error is below. Does anyone know why I am getting this?
WARNING: Use "yarn jar" to launch YARN applications.
16/01/13 20:33:06 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
16/01/13 20:33:06 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
16/01/13 20:33:06 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType
2016-01-13 20:33:06,502 [main] INFO org.apache.pig.Main - Apache Pig version 0.15.0.2.3.2.0-2950 (rexported) compiled Sep 30 2015, 19:39:20
2016-01-13 20:33:06,502 [main] INFO org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/hue/appcache/application_1452469761257_0020/container_e03_1452469761257_0020_01_000002/pig_1452717186500.log
2016-01-13 20:33:07,183 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/yarn/.pigbootup not found
2016-01-13 20:33:07,294 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://sandbox.hortonworks.com:8020
2016-01-13 20:33:08,734 [main] WARN org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2016-01-13 20:33:08,770 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083
2016-01-13 20:33:08,823 [main] INFO hive.metastore - Connected to metastore.
2016-01-13 20:33:09,127 [main] ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. Could not resolve pigstorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Failed to parse: Pig script failed to parse:
<file script.pig, line 9, column 46> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve pigstorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1735)
at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1443)
at org.apache.pig.PigServer.parseAndBuild(PigServer.java:387)
at org.apache.pig.PigServer.executeBatch(PigServer.java:412)
at org.apache.pig.PigServer.executeBatch(PigServer.java:398)
at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:171)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:234)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
at org.apache.pig.Main.run(Main.java:502)
at org.apache.pig.Main.main(Main.java:177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by:
<file script.pig, line 9, column 46> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve pigstorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1339)
at org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1324)
at org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5184)
at org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenerator.java:7782)
at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1669)
at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)
... 17 more
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve pigstorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
at org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:677)
at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1336)
... 25 more
2016-01-13 20:33:09,131 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve pigstorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Details at logfile: /hadoop/yarn/local/usercache/hue/appcache/application_1452469761257_0020/container_e03_1452469761257_0020_01_000002/pig_1452717186500.log
2016-01-13 20:33:09,153 [main] INFO org.apache.pig.Main - Pig script completed in 2 seconds and 779 milliseconds (2779 ms)
