Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

java.lang.ClassNotFoundException: org.apache.hbase.thirdparty.com.google.common.cache.CacheLoader

java.lang.ClassNotFoundException: org.apache.hbase.thirdparty.com.google.common.cache.CacheLoader

New Contributor

Hi,

I am upgrading from hdp2.6.3 to hdp3.1.4. Post upgrade am getting the following error when running one of my jobs

 

Caused by: java.lang.ClassNotFoundException: org.apache.hbase.thirdparty.com.google.common.cache.CacheLoader
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357



I see that the jar containing this class is hbase-shaded-miscellaneous-2.2.0.jar and is present in 

/usr/hdp/3.1.4.0-315/hbase/lib/hbase-shaded-miscellaneous-2.2.0.jar

/usr/hdp/3.1.4.0-315/spark2/jars/hbase-shaded-miscellaneous-2.2.0.jar

 

I can see that the spark2-hdp-yarn-archive.tar.gz created in /hadoop/yarn/local/filecache/0/11837/spark2-hdp-yarn-archive.tar.gz does not contain this jar file.

 

When I add the jar explicity to my spark-submit command by appending it to --jars option and adding it to conf as below , I dont see this error ,

--conf "spark.driver.extraClassPath=/usr/hdp/3.1.4.0-315/spark2/jars/hbase-protocol-shaded-2.0.2.3.1.4.0-315.jar

 

 

Progressing in above manner I then get another ClassNotfound for another shaded class in hbase-protocol-shaded-2.0.2.3.1.4.0-315.jar. 

 

Can I get some pointers on what the issue may be.

Other jars like hbase-client-2.0.2.3.1.4.0-315.jar etc are not giving this issue.

 

Thanks,

Kevin Mathrani

 

1 REPLY 1
Highlighted

Re: java.lang.ClassNotFoundException: org.apache.hbase.thirdparty.com.google.common.cache.CacheLoader

Get all the classes listed as part of error(ClassNotfound) and add those corrresponding jars to spark.driver.extraClassPath and spark.executor.extraClassPath.

 

It worked for us, when all the jars corresponding the class in the error was added.

Don't have an account?
Coming from Hortonworks? Activate your account here