question Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial.. in Archives of Support Questions (Read Only)

Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

prakashpunj — Fri, 19 Feb 2016 13:59:37 GMT

2016-02-19 02:29:21,118 [main] INFO  org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/admin/appcache/application_1455824286177_0003/container_e03_1455824286177_0003_01_000002/pig_1455848961114.log 
 2016-02-19 02:29:22,717 [main] ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. File –useHCatalog does not exist

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

rushikeshdeshmu — Fri, 19 Feb 2016 14:13:41 GMT

@Prakash Punj, Have you checked below link?:

http://stackoverflow.com/questions/24370338/filenotfoundexception-with-pig

Hope it helps you solving this ERROR?

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

bleonhardi — Fri, 19 Feb 2016 17:40:02 GMT

Just one question, did you cut and paste the command? Word has the bad habit of replacing the - sign with a different character that looks the same. Can you verify that the - is really the normal linux command line - ?

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

aervits — Fri, 19 Feb 2016 19:17:19 GMT

@Prakash Punj

you're most likely encountering this problem please see https://community.hortonworks.com/questions/8518/hcatloader-error.html

There's a known bug in sandbox

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

amiller — Sat, 20 Feb 2016 02:30:10 GMT

Confirmed, the pasted character is not a minus sign (ASCII 45 in decimal). You can verify with the 'od' command.

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

aervits — Tue, 23 Feb 2016 10:41:23 GMT

@Prakash Punj use the following

org.apache.hive.hcatalog.pig.HCatLoader

instead of 

org.apache.hcatalog.pig.HCatLoader

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

prakashpunj — Tue, 23 Feb 2016 11:33:25 GMT

@Artem Ervits

Below is the code that I m using in pig script. 

a = LOAD 'geolocation' using org.apache.hive.hcatalog.pig.HCatLoader();
b = filter a by event != 'normal';
c = foreach b generate driverid, event, (int) '1' as occurance;
d = group c by driverid;
e = foreach d generate group as driverid, SUM(c.occurance) as t_occ;
g = LOAD 'drivermileage' using org.apache.hive.hcatalog.pig.HCatLoader();
h = join e by driverid, g by driverid;
final_data = foreach h generate $0 as driverid, $1 as events, $3 as totmiles, (float) $3/$1 as riskfactor;
store final_data into 'riskfactor' using org.apache.hive.hcatalog.pig.HCatStorer();

ERROR that I AM GETTING IS:

ARNING: Use "yarn jar" to launch YARN applications. 
 16/02/23 03:31:35 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL 
 16/02/23 03:31:35 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE 
 16/02/23 03:31:35 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType 
 2016-02-23 03:31:35,609 [main] INFO  org.apache.pig.Main - Apache Pig version 0.15.0.2.3.4.0-3485 (rexported) compiled Dec 16 2015, 04:30:33 
 2016-02-23 03:31:35,609 [main] INFO  org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/admin/appcache/application_1455824286177_0013/container_e03_1455824286177_0013_01_000002/pig_1456198295607.log 
 2016-02-23 03:31:38,312 [main] ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. File –useHCatalog does not exist 
 Details at logfile: /hadoop/yarn/local/usercache/admin/appcache/application_1455824286177_0013/container_e03_1455824286177_0013_01_000002/pig_1456198295607.log 
 2016-02-23 03:31:38,339 [main] INFO  org.apache.pig.Main - Pig script completed in 3 seconds and 412 milliseconds (3412 ms)

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

aervits — Tue, 23 Feb 2016 19:00:22 GMT

You are executing pig script incorrectly, it accepts your -useHCatalog as filename. You can use the pig Ambari view and type -useHCatalog in additional parameters or run command in the terminal. Documentation https://cwiki.apache.org/confluence/display/Hive/HCatalog+LoadStore#HCatalogLoadStore-The-useHCatalogFlag

pig -x tez -useHCatalog filename

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

amiller — Thu, 25 Feb 2016 04:01:38 GMT

@Prakash Punj

Can you verify whether you are still experiencing an issue after replacing the hyphen/dash/minus symbol with the correct character, as described by Benjamin below?

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

prakashpunj — Thu, 25 Feb 2016 07:10:56 GMT

I tried running the command:

pig -x tez -useHCatalog Pigscript but there result is the same. It's saying :

2016-02-24 23:04:54,675 [main] ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. File –useHCatalog does not exist
Details at logfile: /hadoop/yarn/local/usercache/ppunj/appcache/application_1456334730705_0001/container_e05_1456334730705_0001_01_000002/pig_1456355093669.log
2016-02-24 23:04:54,722 [main] INFO  org.apache.pig.Main - Pig script completed in 1 second and 467 milliseconds (1467 ms)

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

nsabharwal — Thu, 25 Feb 2016 09:53:35 GMT

@Prakash Punj

See this thread https://community.hortonworks.com/questions/8723/how-to-make-pig-scripts-should-choose-both-hcatalo.html

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

nsabharwal — Sun, 18 Aug 2019 13:07:29 GMT

See this ..it worked in my env

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

aervits — Thu, 25 Feb 2016 10:10:31 GMT

@Prakash Punj

have you seen my response? What happens if you run your script as I show above?

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

aervits — Thu, 25 Feb 2016 10:11:37 GMT

@Prakash Punj try the following

pig -x tez filename -useHCatalog

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

nsabharwal — Thu, 25 Feb 2016 10:15:51 GMT

@Prakash Punj

pig -useHCatalog -x tez -f p.pig --> this works

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

prakashpunj — Thu, 25 Feb 2016 10:21:24 GMT

@Neeraj Sabharwal

@Artem Ervits

Same error after using the command below.

pig -x tez Pigscript.pig -useHCatalog;


2016-02-25 02:17:33,158 [main] INFO  org.apache.pig.Main - Logging error messages to: /hadoop/yarn/local/usercache/ppunj/appcache/application_1456334730705_0004/container_e05_1456334730705_0004_01_000002/pig_1456366653155.log 
 2016-02-25 02:17:34,257 [main] ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. File –useHCatalog does not exist 
 Details at logfile: /hadoop/yarn/local/usercache/ppunj/appcache/application_1456334730705_0004/container_e05_1456334730705_0004_01_000002/pig_1456366653155.log

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

nsabharwal — Thu, 25 Feb 2016 10:22:06 GMT

@Prakash Punj Read my reply ..

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

nsabharwal — Thu, 25 Feb 2016 10:40:50 GMT

@Prakash Punj make sure you have the adapter in the node

[root@phdns01 ~]# find / -name hive-hcatalog-pig-adapter*.jar

/usr/hdp/2.3.4.0-3485/hive-hcatalog/share/hcatalog/hive-hcatalog-pig-adapter.jar

/usr/hdp/2.3.4.0-3485/hive-hcatalog/share/hcatalog/hive-hcatalog-pig-adapter-1.2.1.2.3.4.0-3485.jar

/usr/hdp/2.3.4.0-3485/oozie/share/lib/pig/hive-hcatalog-pig-adapter-1.2.1.2.3.4.0-3485.jar

[root@phdns01 ~]#

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

prakashpunj — Thu, 25 Feb 2016 10:47:52 GMT

@Neeraj Sabharwal

looks like adapter is there.

[root@hdp-m ~]# find / -name hive-hcatalog-pig-adapter*.jar
/usr/hdp/2.3.4.0-3485/hive-hcatalog/share/hcatalog/hive-hcatalog-pig-adapter-1.2.1.2.3.4.0-3485.jar
/usr/hdp/2.3.4.0-3485/hive-hcatalog/share/hcatalog/hive-hcatalog-pig-adapter.jar
/usr/hdp/2.3.4.0-3485/oozie/share/lib/pig/hive-hcatalog-pig-adapter-1.2.1.2.3.4.0-3485.jar
[root@hdp-m ~]#

Re: Getting –useHCatalog file doesnot exist when running a pig view. This is part of Horton works tutorial..

nsabharwal — Thu, 25 Feb 2016 10:57:16 GMT

@Prakash Punj See my latest reply...You are giving HDFS location for your pig script...it needs to be local file system