I got the same error trying to run Spark on YARN. I fixed it by copying /usr/lib/hadoop/client/hadoop-mapreduce-client-core.jar into HDFS, and then putting that file in my /etc/spark/conf/spark-defaults.conf file for the 'spark.yarn.dist.files' directive:   spark.yarn.dist.files /my/path/on/hdfs/hadoop-mapreduce-client-core.jar ... View more