--deploy-mode option for CDSW

balacs — Tue, 21 Apr 2026 13:27:51 GMT

Is there a recommendation of what --deploy-mode (in spark-submit) option should we use from CDSW engines? If we use --deploy-mode client, does that mean the driver program runs in the CDSW engine?

Thanks !

Re: --deploy-mode option for CDSW

peter_ableda — Wed, 11 Apr 2018 08:33:52 GMT

Hi,

The documentation has an image explaining this:

https://www.cloudera.com/documentation/data-science-workbench/latest/topics/cdsw_dist_comp_with_Spark.html

The answer is yes, if you start a Python 2 session and you create a SparkSession object there you will run the Spark application in client mode and the Spark driver will be inside the CDSW session (docker container).

This is the primary use-case for CDSW.

Regards,

Peter

question Re: --deploy-mode option for CDSW in Archives of Support Questions (Read Only)

--deploy-mode option for CDSW

Re: --deploy-mode option for CDSW