Hi,
I'm getting this error below while trying to submit a scala jar built in Intellij using maven
- spark version 2.3.0
- scala version 2.11.11
Command used:
spark2-submit --master="yarn" --deploy-mode="cluster" --queue root.myyarnqueue --executor-memory 12G --driver-memory 12G --class MyClassName /projects/myscala.jar arg_1 arg_2
Error Message
java.lang.ClassNotFoundException: MyClassName
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.spark.util.Utils$.classForName(Utils.scala:239)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:851)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
20/01/27 15:54:47 INFO util.ShutdownHookManager: Shutdown hook called