Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)

87488-shell9.jpg

87476-shell1.jpg

87478-shell10.jpg

87479-shell3.jpg

87480-shell4.jpg

87481-shell5.jpg

87486-shell11.jpg

87487-shell8.jpg

Why to use spark-submit if spark-shell is there ?


1. Spark shell spawns executors on random nodes and hence chances of data Locality will be very less.
2. Spark-submit based on the Nodes where the data is saved spawns the excutors hence spark-submit will be more performant as compared to spark-shell.

3. Spark shell is good in situations when data exploration needs to be done as it provides a interactive CLI to run your code.


shell6.jpgshell2.jpg
493 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 06:33 AM
Updated by:
 
Contributors
Top Kudoed Authors