Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

ClassNotFoundException when using external library

ClassNotFoundException when using external library



I think this will be an old problem , except for newbies like me.

I have seen some pages which talk about this both on stackoverflow and here


I am using a jar file in my map function, and the jar is accompanied by a .so file.


When I run with just java (no hadoop) it works.

 java  -Djava.library.path=/folder/containing/sharedobjects/ -classpath /externaljars/some.jar:.:/tmp/mypack/   mypack.MyTest


When I try with hadoop it says ClassNotFoundException .

hadoop jar myjar.jar mypack.MyClass  -libjars /externaljars/some.jar  -Djava.library.path=/folder/containing/sharedobjects/
Error: java.lang.ClassNotFoundException:  Someclass
but Someclass is present in some.jar

in my run method I have included this, although it is deprecated:
DistributedCache.addCacheFile(new URI("/folder/containing/sharedobjects/"), conf);

I can use "hdfs://host:port/libraries/", but not sure if I should use localhost:50070.

In my map method I have
 System.load((new File("")).getAbsolutePath());

If someone can reply back with the latest correct usage, it will be very useful. Thank you.


Re: ClassNotFoundException when using external library


I have also copied the third party shared object file , into these folders




I have already tried various methods suggested by various sources, but still looking for a definitive answer which works.



Re: ClassNotFoundException when using external library


I found a workaround, I think.


basically my tool runner was throwing a warning saying cannot read command line options.

I changed the usage slightly , as per


no longer warning, and I think because I copied the shared objects into  lib/native , it sees the shared objects


I think it should also work with command line options now, without having to copy to   lib/native

Don't have an account?
Coming from Hortonworks? Activate your account here