Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

hcat.bin is not defined. Define it to be your hcat script

Solved Go to solution
Highlighted

hcat.bin is not defined. Define it to be your hcat script

Mentor

getting error when executing a pig script through Oozie. Script has pig and hcat commands.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: hcat.bin is not defined. Define it to be your hcat script

Mentor

SOLUTION

Pig introduced an option to run hcatalog commands in grunt and in scripts. There's more info in pig.properties file in /etc/pig/conf/pig.properties

# In addition to the fs-style commands (rm, ls, etc) Pig can now execute

# SQL-style DDL commands, eg "sql create table pig_test(name string, age int)".

# The only implemented backend is hcat, and luckily that's also the default.

#

# pig.sql.type=hcat # Path to the hcat executable, for use with pig.sql.type=hcat (default: null)

#

hcat.bin=/usr/local/hcat/bin/hcat

this is on sandbox 2.3, HDP 2.3 and HDP 2.3.2 by default

so running any pig script with hcat commands without -useHCatalog will fail with the following, usually that will happen through Oozie

Pig Stack Trace --------------- ERROR 2997: Encountered IOException. /usr/local/hcat/bin/hcat does not exist. Please check your 'hcat.bin' setting in pig.properties. java.io.IOException: /usr/local/hcat/bin/hcat does not exist. Please check your 'hcat.bin' setting in pig.properties. at org.apache.pig.tools.grunt.GruntParser.processSQLCommand(GruntParser.java:1286) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81) at org.apache.pig.Main.run(Main.java:631) at org.apache.pig.Main.main(Main.java:177) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

so to fix it change it globally by editing pig.properties in Ambari to point to the following:

hcat.bin=/usr/bin/hcat

or

copy pig.properties to your own location, override the pig.properties with right path (i.e. hcat.bin=/usr/bin/hcat) and execute script like so

pig -P pig.properties test.pig

or

override the property on the fly

pig -Dhcat.bin=/usr/bin/hcat test.pig

Or even lesser intrusive way:

In your pig script put this in the beginning

set hcat.bin /usr/bin/hcat;

4 REPLIES 4

Re: hcat.bin is not defined. Define it to be your hcat script

Mentor

Re: hcat.bin is not defined. Define it to be your hcat script

Mentor

do I need to set $HCAT_HOME/bin/hcat in pig.properties?

Re: hcat.bin is not defined. Define it to be your hcat script

Master Collaborator

That should work I think.

Re: hcat.bin is not defined. Define it to be your hcat script

Mentor

SOLUTION

Pig introduced an option to run hcatalog commands in grunt and in scripts. There's more info in pig.properties file in /etc/pig/conf/pig.properties

# In addition to the fs-style commands (rm, ls, etc) Pig can now execute

# SQL-style DDL commands, eg "sql create table pig_test(name string, age int)".

# The only implemented backend is hcat, and luckily that's also the default.

#

# pig.sql.type=hcat # Path to the hcat executable, for use with pig.sql.type=hcat (default: null)

#

hcat.bin=/usr/local/hcat/bin/hcat

this is on sandbox 2.3, HDP 2.3 and HDP 2.3.2 by default

so running any pig script with hcat commands without -useHCatalog will fail with the following, usually that will happen through Oozie

Pig Stack Trace --------------- ERROR 2997: Encountered IOException. /usr/local/hcat/bin/hcat does not exist. Please check your 'hcat.bin' setting in pig.properties. java.io.IOException: /usr/local/hcat/bin/hcat does not exist. Please check your 'hcat.bin' setting in pig.properties. at org.apache.pig.tools.grunt.GruntParser.processSQLCommand(GruntParser.java:1286) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81) at org.apache.pig.Main.run(Main.java:631) at org.apache.pig.Main.main(Main.java:177) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

so to fix it change it globally by editing pig.properties in Ambari to point to the following:

hcat.bin=/usr/bin/hcat

or

copy pig.properties to your own location, override the pig.properties with right path (i.e. hcat.bin=/usr/bin/hcat) and execute script like so

pig -P pig.properties test.pig

or

override the property on the fly

pig -Dhcat.bin=/usr/bin/hcat test.pig

Or even lesser intrusive way:

In your pig script put this in the beginning

set hcat.bin /usr/bin/hcat;

Don't have an account?
Coming from Hortonworks? Activate your account here