Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

java.lang.NoClassDefFoundError: org/slf4j/impl/StaticLoggerBinder when launching spark2

avatar
New Contributor

I installed cdh 5.11.0, along with spark 2 (according to this doc: https://www.cloudera.com/documentation/spark2/latest/topics/spark2_installing.html). However, when I tried to launch a spark job using spark-submit2, the error below occurred:

 

Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/impl/StaticLoggerBinder
	at org.apache.spark.internal.Logging$class.initializeLogging(Logging.scala:111)
	at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:102)
	at org.apache.spark.deploy.yarn.ApplicationMaster$.initializeLogIfNecessary(ApplicationMaster.scala:746)
	at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
	at org.apache.spark.deploy.yarn.ApplicationMaster$.log(ApplicationMaster.scala:746)
	at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:761)
	at org.apache.spark.deploy.yarn.ExecutorLauncher$.main(ApplicationMaster.scala:795)
	at org.apache.spark.deploy.yarn.ExecutorLauncher.main(ApplicationMaster.scala)
Caused by: java.lang.ClassNotFoundException: org.slf4j.impl.StaticLoggerBinder
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 8 more

 

I realized there was something wrong with the classpath, but can I fix it with the Cloudera Manager? Or should I go and classpath manually on each nodes?

5 REPLIES 5

avatar
Master Collaborator

 

Caused by: java.lang.ClassNotFoundException: org.slf4j.impl.StaticLoggerBinder

  ^ This is generally an indication of a non-existing or incorrect Hadoop/Spark2 client configuration 

 

  I'd make sure that the Spark2 gateway role is added to the node from where you're running spark2-submit.

 

  https://www.cloudera.com/documentation/spark2/latest/topics/spark2_installing.html

  |_When configuring the assignment of role instances to hosts, add a gateway role to every host

 

  If you've already done that, please share the output of: 

  # alternatives --display spark2-conf

  

Example from a working node
# alternatives --display spark2-conf
spark2-conf - status is auto.
 link currently points to /etc/spark2/conf.cloudera.spark2_on_yarn
/opt/cloudera/parcels/SPARK2-2.2.0.cloudera1-1.cdh5.12.0.p0.142354/etc/spark2/conf.dist - priority 10
/etc/spark2/conf.cloudera.spark2_on_yarn - priority 51
Current `best' version is /etc/spark2/conf.cloudera.spark2_on_yarn.

# grep slf4j-log4j /etc/spark2/conf.cloudera.spark2_on_yarn/classpath.txt
/opt/cloudera/parcels/CDH-5.11.0-1.cdh5.11.0.p0.34/jars/slf4j-log4j12-1.7.5.jar

# ls -l /opt/cloudera/parcels/CDH-5.11.0-1.cdh5.11.0.p0.34/jars/slf4j-log4j12-1.7.5.jar
-rw-r--r-- 1 root root 8869 Apr 5 21:44 /opt/cloudera/parcels/CDH-5.11.0-1.cdh5.11.0.p0.34/jars/slf4j-log4j12-1.7.5.jar

 

avatar
New Contributor
I re-added the entire spark2 service and deployed to all hosts. The output of the command 'alternatives --display spark2-conf' is as below: spark2-conf - status is auto. link currently points to /etc/spark2/conf.cloudera.spark2_on_yarn /opt/cloudera/parcels/SPARK2-2.2.0.cloudera1-1.cdh5.12.0.p0.142354/etc/spark2/conf.dist - priority 10 /etc/spark2/conf.cloudera.spark2_on_yarn - priority 51 Current `best' version is /etc/spark2/conf.cloudera.spark2_on_yarn. I only tried to submit a job on one node before I asked the question, so this time I decided to submit the job on several other nodes. The result is that it seems all other nodes is ok expect for the node I used before. The node I failed is where the spark2's history server is running on, the it's not a zookeeper server node. Is this normal? For my understanding, each node should be available for submitting jobs.

avatar
Master Collaborator

Having an SHS (Spark2 History Server) role or not having a ZK (zookeeper) role shouldn't affect the spark job. All we require is a Spark2 gateway role on the node from where you're running the spark2 job. Given other nodes are able to launch the same job, odds are high that we have a problem with client configuration or classpath on this node particularly. BTW, the spark2-conf looks fine.

 

Can you please help confirm if you are able to run a simple spark pi job, or does that fails too with the same message?

 

$ spark2-submit --deploy-mode client --class org.apache.spark.examples.SparkPi /opt/cloudera/parcels/SPARK2/lib/spark2/examples/jars/spark-examples*.jar 10 10

 

17/07/10 04:14:53 INFO spark.SparkContext: Running Spark version 2.1.0.cloudera1

....

Pi is roughly 3.1397831397831397

avatar
Explorer

I copied the missing configuration files from other servers. 

 

[root@servername conf.dist]# ls -ltr
total 52
-rw-r--r-- 1 root root 951 Apr 16 14:50 log4j.properties
-rw-r--r-- 1 root root 148 Apr 16 14:50 __cloudera_metadata__
-rw-r--r-- 1 root root 21 Apr 16 14:50 __cloudera_generation__
-rw-r--r-- 1 root root 12440 Apr 16 14:50 classpath.txt_4172019
-rw-r--r-- 1 root root 12587 Apr 16 14:50 classpath.txt
drwxr-xr-x 2 root root 279 Apr 16 14:50 yarn-conf
-rw-r--r-- 1 root root 2331 Apr 16 14:50 spark-env.sh
-rw-r--r-- 1 root root 1706 Apr 16 14:50 spark-defaults.conf
-rw-r--r-- 1 root root 0 Apr 16 14:50 navigator.lineage.client.properties
[root@servername conf.dist]#

 

After copying above files it worked.

avatar
Explorer

[root@servername conf.dist]# spark2-submit
Usage: spark-submit [options] <app jar | python file | R file> [app arguments]
Usage: spark-submit --kill [submission ID] --master [spark://...]
Usage: spark-submit --status [submission ID] --master [spark://...]
Usage: spark-submit run-example [options] example-class [example args]

Options:
--master MASTER_URL spark://host:port, mesos://host:port, yarn,
k8s://https://host:port, or local (Default: local[*]).
--deploy-mode DEPLOY_MODE Whether to launch the driver program locally ("client") or
on one of the worker machines inside the cluster ("cluster")
(Default: client).
--class CLASS_NAME Your application's main class (for Java / Scala apps).
--name NAME A name of your application.
--jars JARS Comma-separated list of jars to include on the driver
and executor classpaths.
--packages Comma-separated list of maven coordinates of jars to include
on the driver and executor classpaths. Will search the local
maven repo, then maven central and any additional remote
repositories given by --repositories. The format for the
coordinates should be groupId:artifactId:version.
--exclude-packages Comma-separated list of groupId:artifactId, to exclude while
resolving the dependencies provided in --packages to avoid
dependency conflicts.
--repositories Comma-separated list of additional remote repositories to
search for the maven coordinates given with --packages.
--py-files PY_FILES Comma-separated list of .zip, .egg, or .py files to place
on the PYTHONPATH for Python apps.
--files FILES Comma-separated list of files to be placed in the working
directory of each executor. File paths of these files
in executors can be accessed via SparkFiles.get(fileName).

--conf PROP=VALUE Arbitrary Spark configuration property.
--properties-file FILE Path to a file from which to load extra properties. If not
specified, this will look for conf/spark-defaults.conf.

--driver-memory MEM Memory for driver (e.g. 1000M, 2G) (Default: 1024M).
--driver-java-options Extra Java options to pass to the driver.
--driver-library-path Extra library path entries to pass to the driver.
--driver-class-path Extra class path entries to pass to the driver. Note that
jars added with --jars are automatically included in the
classpath.

--executor-memory MEM Memory per executor (e.g. 1000M, 2G) (Default: 1G).

--proxy-user NAME User to impersonate when submitting the application.
This argument does not work with --principal / --keytab.

--help, -h Show this help message and exit.
--verbose, -v Print additional debug output.
--version, Print the version of current Spark.

Cluster deploy mode only:
--driver-cores NUM Number of cores used by the driver, only in cluster mode
(Default: 1).

Spark standalone or Mesos with cluster deploy mode only:
--supervise If given, restarts the driver on failure.
--kill SUBMISSION_ID If given, kills the driver specified.
--status SUBMISSION_ID If given, requests the status of the driver specified.

Spark standalone and Mesos only:
--total-executor-cores NUM Total cores for all executors.

Spark standalone and YARN only:
--executor-cores NUM Number of cores per executor. (Default: 1 in YARN mode,
or all available cores on the worker in standalone mode)

YARN-only:
--queue QUEUE_NAME The YARN queue to submit to (Default: "default").
--num-executors NUM Number of executors to launch (Default: 2).
If dynamic allocation is enabled, the initial number of
executors will be at least NUM.
--archives ARCHIVES Comma separated list of archives to be extracted into the
working directory of each executor.
--principal PRINCIPAL Principal to be used to login to KDC, while running on
secure HDFS.
--keytab KEYTAB The full path to the file that contains the keytab for the
principal specified above. This keytab will be copied to
the node running the Application Master via the Secure
Distributed Cache, for renewing the login tickets and the
delegation tokens periodically.

[root@servername conf.dist]#