Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
Labels (1)
avatar

87488-shell9.jpg

87476-shell1.jpg

87478-shell10.jpg

87479-shell3.jpg

87480-shell4.jpg

87481-shell5.jpg

87486-shell11.jpg

87487-shell8.jpg

Why to use spark-submit if spark-shell is there ?


1. Spark shell spawns executors on random nodes and hence chances of data Locality will be very less.
2. Spark-submit based on the Nodes where the data is saved spawns the excutors hence spark-submit will be more performant as compared to spark-shell.

3. Spark shell is good in situations when data exploration needs to be done as it provides a interactive CLI to run your code.


shell6.jpgshell2.jpg
1,179 Views
Version history
Last update:
‎08-17-2019 06:33 AM
Updated by:
Contributors