Support Questions

Find answers, ask questions, and share your expertise

Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

Rising Star
2016-02-19 02:29:21,118 [main] INFO  org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/admin/appcache/application_1455824286177_0003/container_e03_1455824286177_0003_01_000002/pig_1455848961114.log 
 2016-02-19 02:29:22,717 [main] ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. File –useHCatalog does not exist 
1 ACCEPTED SOLUTION

21 REPLIES 21

@Prakash Punj, Have you checked below link?:

http://stackoverflow.com/questions/24370338/filenotfoundexception-with-pig

Hope it helps you solving this ERROR?

Just one question, did you cut and paste the command? Word has the bad habit of replacing the - sign with a different character that looks the same. Can you verify that the - is really the normal linux command line - ?

Confirmed, the pasted character is not a minus sign (ASCII 45 in decimal). You can verify with the 'od' command.

Mentor
@Prakash Punj

you're most likely encountering this problem please see https://community.hortonworks.com/questions/8518/hcatloader-error.html

There's a known bug in sandbox

Mentor

@Prakash Punj use the following

org.apache.hive.hcatalog.pig.HCatLoader

instead of 

org.apache.hcatalog.pig.HCatLoader

Rising Star
@Artem Ervits
Below is the code that I m using in pig script. 

a = LOAD 'geolocation' using org.apache.hive.hcatalog.pig.HCatLoader();
b = filter a by event != 'normal';
c = foreach b generate driverid, event, (int) '1' as occurance;
d = group c by driverid;
e = foreach d generate group as driverid, SUM(c.occurance) as t_occ;
g = LOAD 'drivermileage' using org.apache.hive.hcatalog.pig.HCatLoader();
h = join e by driverid, g by driverid;
final_data = foreach h generate $0 as driverid, $1 as events, $3 as totmiles, (float) $3/$1 as riskfactor;
store final_data into 'riskfactor' using org.apache.hive.hcatalog.pig.HCatStorer();

ERROR that I AM GETTING IS:

ARNING: Use "yarn jar" to launch YARN applications. 
 16/02/23 03:31:35 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL 
 16/02/23 03:31:35 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE 
 16/02/23 03:31:35 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType 
 2016-02-23 03:31:35,609 [main] INFO  org.apache.pig.Main - Apache Pig version 0.15.0.2.3.4.0-3485 (rexported) compiled Dec 16 2015, 04:30:33 
 2016-02-23 03:31:35,609 [main] INFO  org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/admin/appcache/application_1455824286177_0013/container_e03_1455824286177_0013_01_000002/pig_1456198295607.log 
 2016-02-23 03:31:38,312 [main] ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. File –useHCatalog does not exist 
 Details at logfile: /hadoop/yarn/local/usercache/admin/appcache/application_1455824286177_0013/container_e03_1455824286177_0013_01_000002/pig_1456198295607.log 
 2016-02-23 03:31:38,339 [main] INFO  org.apache.pig.Main - Pig script completed in 3 seconds and 412 milliseconds (3412 ms) 

Mentor

You are executing pig script incorrectly, it accepts your -useHCatalog as filename. You can use the pig Ambari view and type -useHCatalog in additional parameters or run command in the terminal. Documentation https://cwiki.apache.org/confluence/display/Hive/HCatalog+LoadStore#HCatalogLoadStore-The-useHCatalo...

pig -x tez -useHCatalog filename

Mentor
@Prakash Punj

have you seen my response? What happens if you run your script as I show above?

@Prakash Punj

Can you verify whether you are still experiencing an issue after replacing the hyphen/dash/minus symbol with the correct character, as described by Benjamin below?

Rising Star

I tried running the command:

pig -x tez -useHCatalog Pigscript but there result is the same. It's saying :

2016-02-24 23:04:54,675 [main] ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. File –useHCatalog does not exist
Details at logfile: /hadoop/yarn/local/usercache/ppunj/appcache/application_1456334730705_0001/container_e05_1456334730705_0001_01_000002/pig_1456355093669.log
2016-02-24 23:04:54,722 [main] INFO  org.apache.pig.Main - Pig script completed in 1 second and 467 milliseconds (1467 ms)


Mentor

@Prakash Punj try the following

pig -x tez filename -useHCatalog

2380-screen-shot-2016-02-25-at-72543-am.pngSee this ..it worked in my env

@Prakash Punj

pig -useHCatalog -x tez -f p.pig --> this works

@Prakash Punj make sure you have the adapter in the node

[root@phdns01 ~]# find / -name hive-hcatalog-pig-adapter*.jar

/usr/hdp/2.3.4.0-3485/hive-hcatalog/share/hcatalog/hive-hcatalog-pig-adapter.jar

/usr/hdp/2.3.4.0-3485/hive-hcatalog/share/hcatalog/hive-hcatalog-pig-adapter-1.2.1.2.3.4.0-3485.jar

/usr/hdp/2.3.4.0-3485/oozie/share/lib/pig/hive-hcatalog-pig-adapter-1.2.1.2.3.4.0-3485.jar

[root@phdns01 ~]#

Rising Star
@Neeraj Sabharwal

@Artem Ervits

Same error after using the command below.

pig -x tez Pigscript.pig -useHCatalog;


2016-02-25 02:17:33,158 [main] INFO  org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/ppunj/appcache/application_1456334730705_0004/container_e05_1456334730705_0004_01_000002/pig_1456366653155.log 
 2016-02-25 02:17:34,257 [main] ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. File –useHCatalog does not exist 
 Details at logfile: /hadoop/yarn/local/usercache/ppunj/appcache/application_1456334730705_0004/container_e05_1456334730705_0004_01_000002/pig_1456366653155.log 

@Prakash Punj Read my reply ..

Rising Star
@Neeraj Sabharwal

looks like adapter is there.

[root@hdp-m ~]# find / -name hive-hcatalog-pig-adapter*.jar
/usr/hdp/2.3.4.0-3485/hive-hcatalog/share/hcatalog/hive-hcatalog-pig-adapter-1.2.1.2.3.4.0-3485.jar
/usr/hdp/2.3.4.0-3485/hive-hcatalog/share/hcatalog/hive-hcatalog-pig-adapter.jar
/usr/hdp/2.3.4.0-3485/oozie/share/lib/pig/hive-hcatalog-pig-adapter-1.2.1.2.3.4.0-3485.jar
[root@hdp-m ~]#


@Prakash Punj See my latest reply...You are giving HDFS location for your pig script...it needs to be local file system

Rising Star
@Neeraj Sabharwal

Pigscript now works. not sure why it's working now. I tried doing the steps all over again and this time it got excecuted.

So thank you much..

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.