HCatLoader() Error in Load Pig Statement

[#1]

The following Pig Script statement gives error...

A = LOAD 'batting' USING org.apache.hcatalog.pig.HCatLoader();

2016-01-01 19:05:47,197 [main] ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. Could not resolve org.apache.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Failed to parse: Pig script failed to parse: 
<file script.pig, line 1, column 25> pig script failed to validate: org.apache.pig.backend.executionengine.E

Please help.

[#2]

Try starting the pig with -useHCatalog as below

pig -useHCatalog -f yourscript.pig

this can be used for running in the terminal, or for calling a script.

or

Specify the location of the HCatalog jar and add a REGISTER statement with the path of the jar to the top of your script as below

REGISTER /home/user/Installations/hive-0.11.0-bin/hcatalog/share/hcatalog/hcatalog-core-0.11.0.jar;

Please note your path may be different.

[#3]

Gangadhar Thanks for the response. I installed Hartonworks sandbox to learn. what is the default path for Hartonworks sandbox?

[#4]

@rich Hi Rich, please let me know the standard libs we use (RETISTER) in pig scripts on hortonworks sandbox (downloaded from hortonworks site for learning purpose) and also sandbox used for certification.

[#5]

@Vidya SK: You do not need to use the REGISTER command to use any of the standard libraries, which includes the HCatLoader and HCatStorer class.

[#6]

HADOOP_HOME:-/usr/hdp/current/hadoop-client

HIVE_CONF_DIR=/usr/hdp/current/hive-client/conf

HIVE_AUX_JARS_PATH=/usr/hdp/current/hive-webhcat/share/hcatalog/hive-hcatalog-core.jar

[#7]

@rich

Rich, you mentioned that we DO NOT need to REGISTER any libraries for HCatLoader. Then why I am getting the error I mentioned in the first post?

[#8]

Because you are using the wrong class name for HCatLoader. It should be:

org.apache.hive.hcatalog.pig.HCatLoader

Let me know if that fixes your issue.

[#9 · Accepted solution]

Because you are using the wrong class name for HCatLoader. It should be:

org.apache.hive.hcatalog.pig.HCatLoader

Let me know if that fixes your issue.

[#10]

@rich

I ran the following script and get the following error:

A = LOAD 'batting' USING org.apache.hive.hcatalog.pig.HCatLoader(); dump A;

2016-01-03 03:52:33,153 [main] ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. Could not resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Failed to parse: Pig script failed to parse: 
<file script.pig, line 11, column 25> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]

[#11]

I ran the following script and get the following error:

A = LOAD 'batting' USING org.apache.hive.hcatalog.pig.HCatLoader(); dump A;

1. 2016-01-0303:52:33,153[main] ERROR org.apache.pig.PigServer- exception during parsing:Error during parsing.Couldnot resolve org.apache.hive.hcatalog.pig.HCatLoaderusing imports:[, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
2. Failed to parse:Pig script failed to parse:
3. <file script.pig, line 11, column 25> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070:Couldnot resolve org.apache.hive.hcatalog.pig.HCatLoaderusing imports:[, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]

[#12]

Are you running Pig (or starting the Grunt shell) with the -useHCatalog flag?

[#13]

@rich It is working from the Grunt Shell without REGISTER, if I do: $pig -useHCatalog -f myscript.pig

Thanks.

But I am not able to execute the same script from Ambari. Do I have to REGISTER the jars, if I have to execute from Ambari?

In certification, do I have to execute the pig script from Grunt shell or from ambari?

[#14]

How you run the script is irrelevant on the exam. If you can get it to work from the command line, then that is perfect.

[#15]

@rich Thanks Rich. That is very useful info. I can execute from Grunt shell.

As Ambari provides good interface which helps in syntax for script writing, if you can provide any help on running from Ambari would be great.

[#16]

You need to view this page carefully - the current exam is on HDP 2.2 with Ambari 1.7:

http://hortonworks.com/training/class/hdp-certified-developer-hdpcd-exam/

That can change at any time, but for now you should be prepared for that environment. The Sandbox can be helpful when practicing and learning Pig and Hive, but I would highly recommend you attempt the practice exam. It is on the same environment as the real exam and contains tasks similar to the real exam.

[#17]

@Vidya SK are you still having issues with this? Can you accept best answer or provide your own solution?