Error with using 'Store' (Pig)

Rebel — Thu, 14 Jan 2016 04:49:16 GMT

Hi,

Hoping someone can advise. I am running a few lines of Pig code where I get an error.

When I run the four lines of code below in Pig it works fine:

allairports = LOAD 'default.airport' USING org.apache.hive.hcatalog.pig.HCatLoader();

gbairports = filter allairports by iso_country == 'GB';

gbcols = foreach gbairports generate $13, $3, $6;

dump gbcols;

However, when I replace the last line (as below) with a store statement, I get an error.

allairports = LOAD 'default.airport' USING org.apache.hive.hcatalog.pig.HCatLoader();

gbairports = filter allairports by iso_country == 'GB';

gbcols = foreach gbairports generate $13, $3, $6;

store gbcols into '/user/hue/tutorials' using pigstorage(',');

The error is below. Does anyone know why I am getting this?

WARNING: Use "yarn jar" to launch YARN applications.
16/01/13 20:33:06 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
16/01/13 20:33:06 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
16/01/13 20:33:06 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType
2016-01-13 20:33:06,502 [main] INFO  org.apache.pig.Main - Apache Pig version 0.15.0.2.3.2.0-2950 (rexported) compiled Sep 30 2015, 19:39:20
2016-01-13 20:33:06,502 [main] INFO  org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/hue/appcache/application_1452469761257_0020/container_e03_1452469761257_0020_01_000002/pig_1452717186500.log
2016-01-13 20:33:07,183 [main] INFO  org.apache.pig.impl.util.Utils - Default bootup file /home/yarn/.pigbootup not found
2016-01-13 20:33:07,294 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://sandbox.hortonworks.com:8020
2016-01-13 20:33:08,734 [main] WARN  org.apache.hadoop.hive.conf.HiveConf - HiveConf of name hive.metastore.local does not exist
2016-01-13 20:33:08,770 [main] INFO  hive.metastore - Trying to connect to metastore with URI thrift://sandbox.hortonworks.com:9083
2016-01-13 20:33:08,823 [main] INFO  hive.metastore - Connected to metastore.
2016-01-13 20:33:09,127 [main] ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. Could not resolve pigstorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Failed to parse: Pig script failed to parse: 
<file script.pig, line 9, column 46> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve pigstorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
	at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1735)
	at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1443)
	at org.apache.pig.PigServer.parseAndBuild(PigServer.java:387)
	at org.apache.pig.PigServer.executeBatch(PigServer.java:412)
	at org.apache.pig.PigServer.executeBatch(PigServer.java:398)
	at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:171)
	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:234)
	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
	at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
	at org.apache.pig.Main.run(Main.java:502)
	at org.apache.pig.Main.main(Main.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: 
<file script.pig, line 9, column 46> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve pigstorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
	at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1339)
	at org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1324)
	at org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5184)
	at org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenerator.java:7782)
	at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1669)
	at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
	at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
	at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)
	... 17 more
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve pigstorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
	at org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:677)
	at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1336)
	... 25 more
2016-01-13 20:33:09,131 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve pigstorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Details at logfile: /hadoop/yarn/local/usercache/hue/appcache/application_1452469761257_0020/container_e03_1452469761257_0020_01_000002/pig_1452717186500.log

2016-01-13 20:33:09,153 [main] INFO org.apache.pig.Main - Pig script completed in 2 seconds and 779 milliseconds (2779 ms)

Re: Error with using 'Store' (Pig)

daniel_hendrix — Thu, 14 Jan 2016 04:56:10 GMT

I believe it is a matter of case-sensitivity. Please try PigStorage instead of pigstorage.

Re: Error with using 'Store' (Pig)

Rebel — Thu, 14 Jan 2016 05:23:24 GMT

Thanks Daniel, that works!

Re: Error with using 'Store' (Pig)

TimothySpann — Wed, 08 Jun 2016 06:31:06 GMT

Found it

http://hortonworks.com/hadoop-tutorial/how-to-use-hcatalog-basic-pig-hive-commands/#explore-pig-latin-data-transformation

5.4 Save the script and execute it

First you need to add the -useHCatalog (Case Sensitive) argument using the box box in the bottom right hand corner

At the top of the screen, make sure the box “Execute on Tez” is checked. Then click Execute to run the script. This action creates one or more Tez jobs.

@Revlin Abbi Make sure you add the useHCatalog argument

question Re: Error with using 'Store' (Pig) in Archives of Support Questions (Read Only)

Error with using 'Store' (Pig)

Re: Error with using 'Store' (Pig)

Re: Error with using 'Store' (Pig)

Re: Error with using 'Store' (Pig)

5.4 Save the script and execute it