question Re: Default Spark env config error in HDP2.4 installation in Support Questions

Default Spark env config error in HDP2.4 installation

jr_chio — Fri, 16 Sep 2022 10:17:22 GMT

Default Spark env config error in HDP2.4 installation:

#SPARK_DRIVER_MEMORY="512 Mb"

This line include a wrong config param "512 Mb", it should be "512M" without space. This will cause spark-shell yarn client start error:

Invalid initial heap size.

Suggest to update the template.

Re: Default Spark env config error in HDP2.4 installation

jyadav — Mon, 09 May 2016 20:04:17 GMT

@JR Cao

Hi, I think that line is commented out so won't cause any issue by default and YES if user uncommented that line then it will cause error. But I will also share this to spark team to change the default template.

Thanks for the feedback.

Re: Default Spark env config error in HDP2.4 installation

jyadav — Wed, 11 May 2016 16:14:58 GMT

@JR Cao

I have raised a internal HDP Spark jira BUG-57627 for this issue. Thanks for raising this.

Re: Default Spark env config error in HDP2.4 installation

aervits — Thu, 12 May 2016 00:42:29 GMT

@vshukla

Re: Default Spark env config error in HDP2.4 installation

jyadav — Thu, 12 May 2016 23:18:41 GMT

@JR Cao please accept my post as an answer if you are good with the provided information.