Hi just to follow up on this, I have now solved the problem. There were two things that I needed to do: 1. In addition to adding oozie.libpath to my job.properties, I also needed to include oozie.use.system.libpath=true 2. Before I was using the following line to add files to the DistributedCache: FileStatus[] status = fs.listStatus("/application/lib");
if (status != null) {
for (int i = 0; i < status.length; ++i) {
if (!status[i].isDir()) {
DistributedCache.addFileToClassPath(status[i].getPath(), job.getConfiguration(), fs);
}
}
} This appeared to be causing a classpath issue because it was adding hdfs://hostname before the hdfs path. Now I am using the following to remove that and only add the absolute hdfs path: FileStatus[] status = fs.listStatus("/application/lib");
if (status != null) {
for (int i = 0; i < status.length; ++i) {
if (!status[i].isDir()) { Path distCachePath = new Path(status[i].getPath().toUri().getPath());
DistributedCache.addFileToClassPath(distCachePath, job.getConfiguration(), fs);
}
}
} Thankyou to those that replied to my original query for pointing me in the right direction. Andrew
... View more
