question Spark specific recommendations for configuring/tuning a cluster? in Archives of Support Questions (Read Only)

Spark specific recommendations for configuring/tuning a cluster?

ldaluz — Fri, 13 Nov 2015 14:15:49 GMT

What are the recommended configurations and suggested tuning parameters when building a cluster for Spark workloads?

Do we have a best practices guide around this?

Re: Spark specific recommendations for configuring/tuning a cluster?

gbraccialli3 — Mon, 16 Nov 2015 20:39:44 GMT

@Laurence Da Luz

Check these links:

https://spark.apache.org/docs/latest/tuning.html

http://www.slideshare.net/SparkSummit/deep-dive-into-project-tungsten-josh-rosen

http://www.slideshare.net/cfregly/advanced-apache-spark-meetup-project-tungsten-nov-12-2015

http://www.slideshare.net/SparkSummit/building-debugging-and-tuning-spark-machine-leaning-pipelinesjoseph-bradley

http://www.slideshare.net/SparkSummit/04-huang-duan-1

http://www.slideshare.net/SparkSummit/making-sense-of-spark-performancekay-ousterhout

http://www.slideshare.net/SparkSummit/data-storage-tips-for-optimal-spark-performancevida-ha-databricks

Re: Spark specific recommendations for configuring/tuning a cluster?

awatson — Wed, 09 Dec 2015 01:31:06 GMT

@Laurence Da Luz Are you talking about Spark as a whole (e.g. Spark core)? or SparkSQL? Either way @Guilherme Braccialli links seem to hit all the key topics.

Re: Spark specific recommendations for configuring/tuning a cluster?

aervits — Tue, 02 Feb 2016 09:48:39 GMT

@Laurence Da Luz can you accept the best answer to close this thread?