I re-added the entire spark2 service and deployed to all hosts. The output of the command 'alternatives --display spark2-conf' is as below: spark2-conf - status is auto. link currently points to /etc/spark2/conf.cloudera.spark2_on_yarn /opt/cloudera/parcels/SPARK2-2.2.0.cloudera1-1.cdh5.12.0.p0.142354/etc/spark2/conf.dist - priority 10 /etc/spark2/conf.cloudera.spark2_on_yarn - priority 51 Current `best' version is /etc/spark2/conf.cloudera.spark2_on_yarn. I only tried to submit a job on one node before I asked the question, so this time I decided to submit the job on several other nodes. The result is that it seems all other nodes is ok expect for the node I used before. The node I failed is where the spark2's history server is running on, the it's not a zookeeper server node. Is this normal? For my understanding, each node should be available for submitting jobs.
... View more
I installed cdh 5.11.0, along with spark 2 (according to this doc: https://www.cloudera.com/documentation/spark2/latest/topics/spark2_installing.html). However, when I tried to launch a spark job using spark-submit2, the error below occurred: Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/impl/StaticLoggerBinder
Caused by: java.lang.ClassNotFoundException: org.slf4j.impl.StaticLoggerBinder
... 8 more I realized there was something wrong with the classpath, but can I fix it with the Cloudera Manager? Or should I go and classpath manually on each nodes?
... View more