Getting error during Pig script execution in sandbox

[#1]

Hi, I am new to Hadoop ecosystem and am trying my hands on Hortonworks Sandbox for VirtualBox.

I have created a table in Hive (geolocation). Now I am trying to call it in pig as directed by sandbox tutorial by using following command:

a = LOAD 'geolocation' using org.apache.hive.hcatalog.pig.HCatLoader();

When I execute the above statement I get an error. Below is the log :

WARNING: Use "yarn jar" to launch YARN applications.
16/07/22 09:49:17 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
16/07/22 09:49:17 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE
16/07/22 09:49:17 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType
2016-07-22 09:49:17,137 [main] INFO  org.apache.pig.Main - Apache Pig version 0.15.0.2.4.0.0-169 (rexported) compiled Feb 10 2016, 07:50:04
2016-07-22 09:49:17,137 [main] INFO  org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/maria_dev/appcache/application_1469172926970_0005/container_e05_1469172926970_0005_01_000002/pig_1469180957135.log
2016-07-22 09:49:19,165 [main] INFO  org.apache.pig.impl.util.Utils - Default bootup file /home/yarn/.pigbootup not found
2016-07-22 09:49:19,403 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://sandbox.hortonworks.com:8020
2016-07-22 09:49:20,107 [main] INFO  org.apache.pig.PigServer - Pig Script ID for the session: PIG-script.pig-51f4a252-0662-4b8b-9a5c-bac29c0d8878
2016-07-22 09:49:21,075 [main] INFO  org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
2016-07-22 09:49:21,410 [main] INFO  org.apache.pig.backend.hadoop.ATSService - Created ATS Hook
2016-07-22 09:49:21,854 [main] ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. Could not resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Failed to parse: Pig script failed to parse: 
<file script.pig, line 1, column 29> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
	at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1776)
	at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1484)
	at org.apache.pig.PigServer.parseAndBuild(PigServer.java:428)
	at org.apache.pig.PigServer.executeBatch(PigServer.java:453)
	at org.apache.pig.PigServer.executeBatch(PigServer.java:439)
	at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:171)
	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:234)
	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
	at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
	at org.apache.pig.Main.run(Main.java:502)
	at org.apache.pig.Main.main(Main.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: 
<file script.pig, line 1, column 29> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
	at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1339)
	at org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1324)
	at org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5184)
	at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3515)
	at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
	at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
	at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
	at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
	at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)
	... 17 more
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
	at org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:677)
	at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1336)
	... 25 more
2016-07-22 09:49:21,862 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Details at logfile: /hadoop/yarn/local/usercache/maria_dev/appcache/application_1469172926970_0005/container_e05_1469172926970_0005_01_000002/pig_1469180957135.log
2016-07-22 09:49:21,887 [main] INFO  org.apache.pig.Main - Pig script completed in 4 seconds and 972 milliseconds (4972 ms)

Close

Kindly help me with this issue.

[#2 · Accepted solution]

Are you using the latest sandbox? Your statement is correct if your pig version is 0.14+. In latest sandbox you should be okay which may mean you are not invoking hcatalog. are you executing from pig ambari view? If so, please check the box for -useHCatalog. In the grunt shell you need to pass the hcatalog parameter, there are a few ways of doing that. One way is to start pig grunt shell with

pig -useHCatalog

[#3]

Hi Artem. I was able to solve the issue by adding -usHCatalog in arguments. I wonder why they have not given this information before execution of the script in the tutorial. I found it in the recap section.

Thanks! 🙂

[#4]

Excellent, I will notify the tutorials team, what is the tutorial number and title?

[#5]

This is the link to the tutorial: https://github.com/hortonworks/tutorials/blob/hdp-2.5/tutorials/hortonworks/hello-hdp-an-introductio...