Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Removing a Jar from Spark default classpath

Highlighted

Removing a Jar from Spark default classpath

New Contributor

Hi, I am running a spark application but I noticed the spark adding two version of joda based on different locations from my cluster while running my Spark application on the cluster (HDP 2.6.2)

one is: /usr/hdp/current/hadoop-client/lib/joda-time-2.9.4.jarSystem Classpath

another is: /hadoop/yarn/local/usercache/livy/appcache/application_1506580942100_2367/container_e55_1506580942100_2367_01_000002/__spark_libs__/joda-time-2.9.3.jarSystem Classpath

This caused my application give me below exception in Spark application

17/11/06 12:01:39 ERROR SparkUncaughtExceptionHandler: Uncaught exception in thread Thread[H2O Launcher thread,5,main] java.lang.NoSuchMethodError: org.joda.time.DateTime.now()Lorg/joda/time/DateTime; at water.util.Timer.nowAsLogString(Timer.java:38) at water.util.Log.header(Log.java:163) at water.util.Log.write0(Log.java:131) at water.util.Log.write0(Log.java:124) at water.util.Log.write(Log.java:109) at water.util.Log.log(Log.java:86) at water.util.Log.info(Log.java:72) at water.H2OSecurityManager.<init>(H2OSecurityManager.java:57) at water.H2OSecurityManager.instance(H2OSecurityManager.java:79) at water.H2ONode.<init>(H2ONode.java:127)

Not sure how to deal with with above exception. Can anyone give me an advise?