Spark specific recommendations for configuring/tuning a cluster?

[#1]

What are the recommended configurations and suggested tuning parameters when building a cluster for Spark workloads?

Do we have a best practices guide around this?

[#2 · Accepted solution]

@Laurence Da Luz

Check these links:

https://spark.apache.org/docs/latest/tuning.html

http://www.slideshare.net/SparkSummit/deep-dive-into-project-tungsten-josh-rosen

http://www.slideshare.net/cfregly/advanced-apache-spark-meetup-project-tungsten-nov-12-2015

http://www.slideshare.net/SparkSummit/building-debugging-and-tuning-spark-machine-leaning-pipelinesj...

http://www.slideshare.net/SparkSummit/04-huang-duan-1

http://www.slideshare.net/SparkSummit/making-sense-of-spark-performancekay-ousterhout

http://www.slideshare.net/SparkSummit/data-storage-tips-for-optimal-spark-performancevida-ha-databri...

[#3]

@Laurence Da Luz Are you talking about Spark as a whole (e.g. Spark core)? or SparkSQL? Either way @Guilherme Braccialli links seem to hit all the key topics.

[#4]

@Laurence Da Luz can you accept the best answer to close this thread?