New Contributor
Posts: 1
Registered: ‎04-05-2018
Accepted Solution

--deploy-mode option for CDSW

Is there a recommendation of what --deploy-mode (in spark-submit) option should we use from CDSW engines? If we use --deploy-mode client, does that mean the driver program runs in the CDSW engine? 


Thanks !

Cloudera Employee
Posts: 43
Registered: ‎07-09-2015

Re: --deploy-mode option for CDSW



The documentation has an image explaining this:


The answer is yes, if you start a Python 2 session and you create a SparkSession object there you will run the Spark application in client mode and the Spark driver will be inside the CDSW session (docker container). 

This is the primary use-case for CDSW.