https://community.cloudera.com/t5/Support-Questions/How-to-pick-number-of-executors-cores-for-each-executor-and/m-p/194941#M157000 <P>Cloudera has a nice two part tuning guide. Attaching the links: </P><UL><LI><A href="http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-1/" target="_blank">http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-1/</A></LI><LI><A href="http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-2/" target="_blank">http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-2/</A></LI></UL> Tue, 23 Jan 2018 23:02:38 GMT sivaprasanna246 2018-01-23T23:02:38Z https://community.cloudera.com/t5/Support-Questions/How-to-pick-number-of-executors-cores-for-each-executor-and/m-p/194940#M156999 <P>Cluster Information:</P><P>10 Node cluster, each machine has 16 cores and 126.04 GB of RAM</P><P>My Question how to pick num-executors, executor-memory, executor-core, driver-memory, driver-cores</P><P>Job will run using Yarn as resource schdeuler </P> Mon, 22 Jan 2018 18:37:54 GMT https://community.cloudera.com/t5/Support-Questions/How-to-pick-number-of-executors-cores-for-each-executor-and/m-p/194940#M156999 pranay_bomminen 2018-01-22T18:37:54Z https://community.cloudera.com/t5/Support-Questions/How-to-pick-number-of-executors-cores-for-each-executor-and/m-p/194941#M157000 <P>Cloudera has a nice two part tuning guide. Attaching the links: </P><UL><LI><A href="http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-1/" target="_blank">http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-1/</A></LI><LI><A href="http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-2/" target="_blank">http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-2/</A></LI></UL> Tue, 23 Jan 2018 23:02:38 GMT https://community.cloudera.com/t5/Support-Questions/How-to-pick-number-of-executors-cores-for-each-executor-and/m-p/194941#M157000 sivaprasanna246 2018-01-23T23:02:38Z https://community.cloudera.com/t5/Support-Questions/How-to-pick-number-of-executors-cores-for-each-executor-and/m-p/194942#M157001 <P>@Sivaprasanna - cloudera blog share above is not available, the link redirects to <A href="https://blog.cloudera.com/">https://blog.cloudera.com/</A> any idea?</P> Fri, 02 Aug 2019 17:12:20 GMT https://community.cloudera.com/t5/Support-Questions/How-to-pick-number-of-executors-cores-for-each-executor-and/m-p/194942#M157001 srinivasachari_ 2019-08-02T17:12:20Z https://community.cloudera.com/t5/Support-Questions/How-to-pick-number-of-executors-cores-for-each-executor-and/m-p/194943#M157002 <P><A rel="user" href="#">@srinivas Nalanagula</A></P><P><A rel="noopener noreferrer noopener noreferrer noopener noreferrer noopener noreferrer" href="https://blog.cloudera.com/how-to-tune-your-apache-spark-jobs-part-2/" target="_blank">https://blog.cloudera.com/how-to-tune-your-apache-spark-jobs-part-2/</A></P><P><A rel="noopener noreferrer noopener noreferrer noopener noreferrer noopener noreferrer" href="https://blog.cloudera.com/how-to-tune-your-apache-spark-jobs-part-1/" target="_blank">https://blog.cloudera.com/how-to-tune-your-apache-spark-jobs-part-1/</A></P> Mon, 12 Aug 2019 15:08:33 GMT https://community.cloudera.com/t5/Support-Questions/How-to-pick-number-of-executors-cores-for-each-executor-and/m-p/194943#M157002 deppfx 2019-08-12T15:08:33Z https://community.cloudera.com/t5/Support-Questions/How-to-pick-number-of-executors-cores-for-each-executor-and/m-p/194944#M157003 <P><A rel="user" href="#">@srinivas Nalanagula</A></P><P><A rel="noopener noreferrer noopener noreferrer" href="https://blog.cloudera.com/how-to-tune-your-apache-spark-jobs-part-2/" target="_blank">https://blog.cloudera.com/how-to-tune-your-apache-spark-jobs-part-2/</A></P><P><A rel="noopener noreferrer noopener noreferrer" href="https://blog.cloudera.com/how-to-tune-your-apache-spark-jobs-part-1/" target="_blank">https://blog.cloudera.com/how-to-tune-your-apache-spark-jobs-part-1/</A></P> Mon, 12 Aug 2019 18:34:23 GMT https://community.cloudera.com/t5/Support-Questions/How-to-pick-number-of-executors-cores-for-each-executor-and/m-p/194944#M157003 deppfx 2019-08-12T18:34:23Z https://community.cloudera.com/t5/Support-Questions/How-to-pick-number-of-executors-cores-for-each-executor-and/m-p/286854#M212703 <P>Hi,</P><P>&nbsp;</P><P>Whether those links that was provided helped to solve the issue</P><P>&nbsp;</P><UL><LI><A href="http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-1/" target="_blank" rel="nofollow noopener noreferrer">http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-1/</A></LI><LI><A href="http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-2/" target="_blank" rel="nofollow noopener noreferrer">http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-2/</A></LI></UL><P>if it was fine can you please mark this forum as solved.</P><P>&nbsp;</P><P>Thanks</P><P>AKR</P> Sun, 05 Jan 2020 12:03:11 GMT https://community.cloudera.com/t5/Support-Questions/How-to-pick-number-of-executors-cores-for-each-executor-and/m-p/286854#M212703 AKR 2020-01-05T12:03:11Z