question Re: HCatLoader() Error in Load Pig Statement in Support Questions

HCatLoader() Error in Load Pig Statement

vidyaranya_kupp — Sat, 02 Jan 2016 05:46:36 GMT

The following Pig Script statement gives error...

A = LOAD 'batting' USING org.apache.hcatalog.pig.HCatLoader();

2016-01-01 19:05:47,197 [main] ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. Could not resolve org.apache.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Failed to parse: Pig script failed to parse: 
<file script.pig, line 1, column 25> pig script failed to validate: org.apache.pig.backend.executionengine.E

Please help.

Re: HCatLoader() Error in Load Pig Statement

GeeKay2015 — Sat, 02 Jan 2016 11:25:45 GMT

Try starting the pig with -useHCatalog as below

pig -useHCatalog -f yourscript.pig

this can be used for running in the terminal, or for calling a script.

Specify the location of the HCatalog jar and add a REGISTER statement with the path of the jar to the top of your script as below

Please note your path may be different.

Re: HCatLoader() Error in Load Pig Statement

vidyaranya_kupp — Sat, 02 Jan 2016 12:14:03 GMT

Gangadhar Thanks for the response. I installed Hartonworks sandbox to learn. what is the default path for Hartonworks sandbox?

Re: HCatLoader() Error in Load Pig Statement

vidyaranya_kupp — Sun, 03 Jan 2016 06:48:16 GMT

@rich Hi Rich, please let me know the standard libs we use (RETISTER) in pig scripts on hortonworks sandbox (downloaded from hortonworks site for learning purpose) and also sandbox used for certification.

Re: HCatLoader() Error in Load Pig Statement

GeeKay2015 — Sun, 03 Jan 2016 08:14:03 GMT

HADOOP_HOME:-/usr/hdp/current/hadoop-client

HIVE_CONF_DIR=/usr/hdp/current/hive-client/conf

HIVE_AUX_JARS_PATH=/usr/hdp/current/hive-webhcat/share/hcatalog/hive-hcatalog-core.jar

Re: HCatLoader() Error in Load Pig Statement

rich1 — Sun, 03 Jan 2016 10:31:44 GMT

@Vidya SK: You do not need to use the REGISTER command to use any of the standard libraries, which includes the HCatLoader and HCatStorer class.

Re: HCatLoader() Error in Load Pig Statement

vidyaranya_kupp — Sun, 03 Jan 2016 11:17:36 GMT

@rich

Rich, you mentioned that we DO NOT need to REGISTER any libraries for HCatLoader. Then why I am getting the error I mentioned in the first post?

Re: HCatLoader() Error in Load Pig Statement

rich1 — Sun, 03 Jan 2016 11:22:13 GMT

Because you are using the wrong class name for HCatLoader. It should be:

org.apache.hive.hcatalog.pig.HCatLoader

Let me know if that fixes your issue.

Re: HCatLoader() Error in Load Pig Statement

rich1 — Sun, 03 Jan 2016 11:22:42 GMT

Because you are using the wrong class name for HCatLoader. It should be:

org.apache.hive.hcatalog.pig.HCatLoader

Let me know if that fixes your issue.

Re: HCatLoader() Error in Load Pig Statement

vidyaranya_kupp — Sun, 03 Jan 2016 11:54:41 GMT

@rich

I ran the following script and get the following error:

A = LOAD 'batting' USING org.apache.hive.hcatalog.pig.HCatLoader(); dump A;

2016-01-03 03:52:33,153 [main] ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. Could not resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Failed to parse: Pig script failed to parse: 
<file script.pig, line 11, column 25> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]

Re: HCatLoader() Error in Load Pig Statement

vidyaranya_kupp — Sun, 03 Jan 2016 11:55:27 GMT

I ran the following script and get the following error:

A = LOAD 'batting' USING org.apache.hive.hcatalog.pig.HCatLoader(); dump A;

2016-01-0303:52:33,153[main] ERROR org.apache.pig.PigServer- exception during parsing:Error during parsing.Couldnot resolve org.apache.hive.hcatalog.pig.HCatLoaderusing imports:[, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Failed to parse:Pig script failed to parse:
<file script.pig, line 11, column 25> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070:Couldnot resolve org.apache.hive.hcatalog.pig.HCatLoaderusing imports:[, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]

Re: HCatLoader() Error in Load Pig Statement

rich1 — Mon, 04 Jan 2016 13:01:46 GMT

Are you running Pig (or starting the Grunt shell) with the -useHCatalog flag?

Re: HCatLoader() Error in Load Pig Statement

vidyaranya_kupp — Tue, 05 Jan 2016 04:37:57 GMT

@rich It is working from the Grunt Shell without REGISTER, if I do: $pig -useHCatalog -f myscript.pig

Thanks.

But I am not able to execute the same script from Ambari. Do I have to REGISTER the jars, if I have to execute from Ambari?

In certification, do I have to execute the pig script from Grunt shell or from ambari?

Re: HCatLoader() Error in Load Pig Statement

rich1 — Tue, 05 Jan 2016 04:41:50 GMT

How you run the script is irrelevant on the exam. If you can get it to work from the command line, then that is perfect.

Re: HCatLoader() Error in Load Pig Statement

vidyaranya_kupp — Tue, 05 Jan 2016 04:48:11 GMT

@rich Thanks Rich. That is very useful info. I can execute from Grunt shell.

As Ambari provides good interface which helps in syntax for script writing, if you can provide any help on running from Ambari would be great.

Re: HCatLoader() Error in Load Pig Statement

rich1 — Tue, 05 Jan 2016 05:11:38 GMT

You need to view this page carefully - the current exam is on HDP 2.2 with Ambari 1.7:

http://hortonworks.com/training/class/hdp-certified-developer-hdpcd-exam/

That can change at any time, but for now you should be prepared for that environment. The Sandbox can be helpful when practicing and learning Pig and Hive, but I would highly recommend you attempt the practice exam. It is on the same environment as the real exam and contains tasks similar to the real exam.

Re: HCatLoader() Error in Load Pig Statement

aervits — Wed, 03 Feb 2016 03:26:48 GMT

@Vidya SK are you still having issues with this? Can you accept best answer or provide your own solution?