Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Path for jar using Oozie

Path for jar using Oozie

Contributor

Hi,

 

When I launch a script shell from Oozie, the workflow fails because it requires jar files that are present on local path but not on hdfs.

 

for instance:

 

 java.io.FileNotFoundException: File does not exist: hdfs://salcls0226.com:8020/cloudera/opt/cloudera/parcels/CDH-5.0.2-1.cdh5.0.2.p0.13/lib/hbase/lib/hbase-server-0.96.1.1-cdh5.0.2.jar

How can I specify that the file hbase-server-0.96.1.1-cdh5.0.2.jar is not present on hdfs but it is in /cloudera/opt/cloudera/parcels/CDH-5.0.2-1.cdh5.0.2.p0.13/lib/hbase/lib/ on local nodes?

 

Thanks 

Alessio

 

 

3 REPLIES 3

Re: Path for jar using Oozie

Master Guru
This form of a confusion would typically happen if your TaskTracker or NodeManager hosts do not have local gateway client configs deployed on them.

Can you add YARN (or MapReduce) gateways to all your hosts, deploy your client configs again, and retry?

Re: Path for jar using Oozie

New Contributor

Still it doesnt work.. I have the same problem while running a shell script. It searches for library in the local path.

Re: Path for jar using Oozie

Master Guru
I'm assuming you've deployed all NMs as gateways for every relevant service (HDFS, YARN/MR1, Hive, etc.) already.

Please try adding the below line to the top of your shell script before you invoke any other commands inside it:

export HADOOP_CONF_DIR=/etc/hadoop/conf

If you are invoking Hive commands, then it would instead be the below:

export HADOOP_CONF_DIR=/etc/hive/conf