Reply
Contributor
Posts: 29
Registered: ‎08-01-2014

Path for jar using Oozie

[ Edited ]

Hi,

 

When I launch a script shell from Oozie, the workflow fails because it requires jar files that are present on local path but not on hdfs.

 

for instance:

 

 java.io.FileNotFoundException: File does not exist: hdfs://salcls0226.com:8020/cloudera/opt/cloudera/parcels/CDH-5.0.2-1.cdh5.0.2.p0.13/lib/hbase/lib/hbase-server-0.96.1.1-cdh5.0.2.jar

How can I specify that the file hbase-server-0.96.1.1-cdh5.0.2.jar is not present on hdfs but it is in /cloudera/opt/cloudera/parcels/CDH-5.0.2-1.cdh5.0.2.p0.13/lib/hbase/lib/ on local nodes?

 

Thanks 

Alessio

 

 

Posts: 1,824
Kudos: 406
Solutions: 292
Registered: ‎07-31-2013

Re: Path for jar using Oozie

This form of a confusion would typically happen if your TaskTracker or NodeManager hosts do not have local gateway client configs deployed on them.

Can you add YARN (or MapReduce) gateways to all your hosts, deploy your client configs again, and retry?
New Contributor
Posts: 2
Registered: ‎06-08-2015

Re: Path for jar using Oozie

Still it doesnt work.. I have the same problem while running a shell script. It searches for library in the local path.

Posts: 1,824
Kudos: 406
Solutions: 292
Registered: ‎07-31-2013

Re: Path for jar using Oozie

I'm assuming you've deployed all NMs as gateways for every relevant service (HDFS, YARN/MR1, Hive, etc.) already.

Please try adding the below line to the top of your shell script before you invoke any other commands inside it:

export HADOOP_CONF_DIR=/etc/hadoop/conf

If you are invoking Hive commands, then it would instead be the below:

export HADOOP_CONF_DIR=/etc/hive/conf
Announcements