I tried to run a spark job on Cloudera by overwriting some basics spark configurations. So I created a spark job with the following Configurations (optional):
spark.eventLog.dir = myPath
spark.eventLog.enabled = true
spark.submit.deployMode = cluster
I have executed this job by running a cde cli command. The submitter keeps my overwriting values for these 3 Spark configurations but these values weren't retrieve in my Spark driver. For instance, I found a "default" tmpPath for spark.eventLog.dir instead of myPath and my driver used a client deployMode.
Some other configurations (like spark.history.fs.logDirectory) were well overwriting at the same time.