Where do custom spark properties end ?

schausson — Thu, 14 Sep 2017 17:18:16 GMT

Hi,

I'm trying to set a default value for "spark.driver.extraJavaOptions" configuration property from ambari, in order to avoid that all my users have to define it in the command line arguments.

I tried to define this property in the "Custom spark-javaopts-properties" section in the ambari UI, but it didn't work (the property seems not to be used anywhere), and even worse, I am not able to find out where this property ends up ? I thought it should be written to a spark configuration file (spark-defaults.conf or anything else), but couldn't find the property anywhere ...

Does anyone know if I picked up the right place to define the property and where it goes in configuration files ?

Thanks for your help

Re: Where do custom spark properties end ?

schausson — Fri, 15 Sep 2017 20:37:28 GMT

In fact I realized that I had to set this properties in the "Custom spark-default" section in Ambari. This way, they are written to spark-defaults.conf configuration file and things work fine

question Re: Where do custom spark properties end ? in Archives of Support Questions (Read Only)

Where do custom spark properties end ?

Re: Where do custom spark properties end ?