I am running Spark using the CDH5 client packages on Ubuntu 12.04. I was trying to get the JavaKafkaWordCount example working in the /usr/lib/spark/examples/lib/spark-examples_2.10-1.0.0-cdh5.1.2.jar running on YARN, but I got an error that I can't seem to resolve (as a side note, I was able to get the SparkPi.scala example to work). I noticed that the spark-streaming-kafka library is not available by default in the CDH5 packages, but I did find that it is available here: https://repository.cloudera.com/cloudera/public/org/apache/spark/spark-streaming-kafka_2.10/1.0.0-cdh5.1.2/. So, I downloaded the spark-streaming-kafka JAR file and I ran the following command:   spark-submit --jars spark-streaming-kafka_2.10-1.0.0-cdh5.1.2.jar --class org.apache.spark.examples.streaming.JavaKafkaWordCount spark-examples_2.10-1.0.0-cdh5.1.2.jar my_kafka_host:my_kafka_port my_consumer_group my_kafka_topic 1     but I got this error:   INFO cluster.YarnClientClusterScheduler: YarnClientClusterScheduler.postStartHook done Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/streaming/kafka/KafkaUtils at org.apache.spark.examples.streaming.JavaKafkaWordCount.main(JavaKafkaWordCount.java:79) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:292) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:55) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.lang.ClassNotFoundException: org.apache.spark.streaming.kafka.KafkaUtils at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 8 more   I also tried adding the spark-streaming-kafka JAR file to the 'spark.yarn.dist.files' directive in my 'spark-defaults.conf' file. That did not work either. Anyone know how to resolve this error? ... View more