I had a similar issue and saw that the spark.yarn.executer.memoryOverhead value was set to a very high value and Cloudera manager picks MB by default but i had put it in bytes . Reverting it back to nil resolved my issue . ... View more