question Re: Is there any specific method to know how much amount of memory we need to allocate for spark configuration like yan-utilities.py? Please suggest how can we tune spark? in Archives of Support Questions (Read Only) https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-there-any-specific-method-to-know-how-much-amount-of/m-p/174697#M37879 <P>@<A href="https://community.hortonworks.com/users/1966/prasad4ucom.html">Ram D</A></P><P>Nothing automated, however, you can c<U>onfigure Dynamic Resource Allocation manually</U>, as one time activity<U>:</U></P><P><A href="http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_spark-guide/content/config-dra-manual.html" target="_blank">http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_spark-guide/content/config-dra-manual.html</A></P><P>Some more here:</P><P><A href="https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_spark-guide/content/ch_tuning-spark.html" target="_blank">https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_spark-guide/content/ch_tuning-spark.html</A></P> Tue, 16 Aug 2016 03:47:21 GMT cstanca 2016-08-16T03:47:21Z Is there any specific method to know how much amount of memory we need to allocate for spark configuration like yan-utilities.py? Please suggest how can we tune spark? https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-there-any-specific-method-to-know-how-much-amount-of/m-p/174696#M37878 Mon, 15 Aug 2016 22:36:45 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-there-any-specific-method-to-know-how-much-amount-of/m-p/174696#M37878 prasad4u_com 2016-08-15T22:36:45Z Re: Is there any specific method to know how much amount of memory we need to allocate for spark configuration like yan-utilities.py? Please suggest how can we tune spark? https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-there-any-specific-method-to-know-how-much-amount-of/m-p/174697#M37879 <P>@<A href="https://community.hortonworks.com/users/1966/prasad4ucom.html">Ram D</A></P><P>Nothing automated, however, you can c<U>onfigure Dynamic Resource Allocation manually</U>, as one time activity<U>:</U></P><P><A href="http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_spark-guide/content/config-dra-manual.html" target="_blank">http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_spark-guide/content/config-dra-manual.html</A></P><P>Some more here:</P><P><A href="https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_spark-guide/content/ch_tuning-spark.html" target="_blank">https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_spark-guide/content/ch_tuning-spark.html</A></P> Tue, 16 Aug 2016 03:47:21 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-there-any-specific-method-to-know-how-much-amount-of/m-p/174697#M37879 cstanca 2016-08-16T03:47:21Z