Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

HCatLoader() Error in Load Pig Statement

avatar
Rising Star

The following Pig Script statement gives error...

A = LOAD 'batting' USING org.apache.hcatalog.pig.HCatLoader();

2016-01-01 19:05:47,197 [main] ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. Could not resolve org.apache.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Failed to parse: Pig script failed to parse: 
<file script.pig, line 1, column 25> pig script failed to validate: org.apache.pig.backend.executionengine.E

Please help.

1 ACCEPTED SOLUTION

avatar
Guru

Because you are using the wrong class name for HCatLoader. It should be:

org.apache.hive.hcatalog.pig.HCatLoader

Let me know if that fixes your issue.

View solution in original post

16 REPLIES 16

avatar
Expert Contributor

Try starting the pig with -useHCatalog as below

pig -useHCatalog -f yourscript.pig

this can be used for running in the terminal, or for calling a script.

or

Specify the location of the HCatalog jar and add a REGISTER statement with the path of the jar to the top of your script as below

REGISTER /home/user/Installations/hive-0.11.0-bin/hcatalog/share/hcatalog/hcatalog-core-0.11.0.jar;

Please note your path may be different.

avatar
Rising Star

Gangadhar Thanks for the response. I installed Hartonworks sandbox to learn. what is the default path for Hartonworks sandbox?

avatar
Rising Star

@rich Hi Rich, please let me know the standard libs we use (RETISTER) in pig scripts on hortonworks sandbox (downloaded from hortonworks site for learning purpose) and also sandbox used for certification.

avatar
Guru

@Vidya SK: You do not need to use the REGISTER command to use any of the standard libraries, which includes the HCatLoader and HCatStorer class.

avatar
Expert Contributor

HADOOP_HOME:-/usr/hdp/current/hadoop-client

HIVE_CONF_DIR=/usr/hdp/current/hive-client/conf

HIVE_AUX_JARS_PATH=/usr/hdp/current/hive-webhcat/share/hcatalog/hive-hcatalog-core.jar

avatar
Rising Star
@rich

Rich, you mentioned that we DO NOT need to REGISTER any libraries for HCatLoader. Then why I am getting the error I mentioned in the first post?

avatar
Guru

Because you are using the wrong class name for HCatLoader. It should be:

org.apache.hive.hcatalog.pig.HCatLoader

Let me know if that fixes your issue.

avatar
Guru

Because you are using the wrong class name for HCatLoader. It should be:

org.apache.hive.hcatalog.pig.HCatLoader

Let me know if that fixes your issue.

avatar
Rising Star

@rich

I ran the following script and get the following error:

A = LOAD 'batting' USING org.apache.hive.hcatalog.pig.HCatLoader(); dump A;

2016-01-03 03:52:33,153 [main] ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. Could not resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Failed to parse: Pig script failed to parse: 
<file script.pig, line 11, column 25> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]