Member since
07-21-2017
3
Posts
0
Kudos Received
0
Solutions
07-25-2017
05:07 PM
Hi @kalai selvan so using the option --driver-class-path has worked - thanks Had to include it and the --jars option still [hdfs@namenode jars]$ spark-submit --class org.apache.spark.examples.streaming.JavaDirectKafkaWordCount --master local[4] --driver-class-path /tmp/spark-streaming-kafka-0-8-assembly_2.11-2.1.1.2.6.1.0-129.jar --jars /tmp/spark-streaming-kafka-0-8-assembly_2.11-2.1.1.2.6.1.0-129.jar spark-examples_2.11-2.1.1.2.6.1.0-129.jar broker1:6667 test2 this does not work though in cluster mode even after copying the jars to all the nodes, same directory, why would this be? Also, which why and when do we need to use --driver-class-path?
... View more