Reply
New Contributor
Posts: 2
Registered: ‎11-22-2017

spark on cloudera quick start vm

How to start spak in master yarn mode  not in local mode .

in cloudera quickstart vm 

Champion
Posts: 601
Registered: ‎05-16-2016

Re: spark on cloudera quick start vm

Could you let me know if your trying to fire the job in python,scala or java ? 

 

 

spark-submit --class YourClassName  \
--master yarn --deploy-mode cluster --executor-memory 1g 
--conf
New Contributor
Posts: 2
Registered: ‎11-22-2017

Re: spark on cloudera quick start vm

I am trying to read a file from hdfs using spark configuration object.
When I am running spark-shell it is running on local mode.please let me now
how to start in yarn mode.


Thanks & Regards
Surya
Expert Contributor
Posts: 152
Registered: ‎07-01-2015

Re: spark on cloudera quick start vm

spark-shell --master yarn-client

Champion
Posts: 601
Registered: ‎05-16-2016

Re: spark on cloudera quick start vm

did you try firing the above command let me know if that works ? 

Announcements