question Re: Overwrite yarn, mapred, tez and hive configurations after restarting services in Ambari in Archives of Support Questions (Read Only)

Overwrite yarn, mapred, tez and hive configurations after restarting services in Ambari

Sanaz_janbakhsh — Sat, 02 Sep 2017 01:41:02 GMT

Hi,

I ran the yarn calculator and it suggested me to update the parameters in mapred-site.xml, yarn-site.xml ,tez-site.xml and hive-site.xml. So i followed the following steps but after starting the services, it overwrites the configurations and i cannot see any changes in the xml files.

- Stop all the servcies through Ambari

- Updated the xml files

- Started all services via Ambari

Any advice?

Thanks,

Re: Overwrite yarn, mapred, tez and hive configurations after restarting services in Ambari

gsaha — Sat, 02 Sep 2017 04:10:38 GMT

- Updated the xml files

For the above step, are you making the changes manually directly to the xml files or via Ambari UI in Configs tab? If you are making the changes outside Ambari, then Ambari always overwrites them when you start services via Ambari. Please make changes via Ambari Configs tab to be able to persist them across restarts.

-Gour

Re: Overwrite yarn, mapred, tez and hive configurations after restarting services in Ambari

Sanaz_janbakhsh — Tue, 05 Sep 2017 13:14:45 GMT

Hi Gour,

Can you please let me know how should i update them in the Ambari config? I cannot find those parameters at all in the config file also i tried to add the manually as a custom yarnsite properties but when for example i tried to add "yarn.scheduler.minimum-allocation-mb", it says that it is already exist which is true ( they are exist in xml file)

SJ.

Re: Overwrite yarn, mapred, tez and hive configurations after restarting services in Ambari

gsaha — Sun, 18 Aug 2019 10:08:02 GMT

yarn.scheduler.minimum-allocation-mb is exposed by Ambari in the "Settings" tab as a slider (shown below) and not in the Advanced tab. That's why you will not find it anywhere in the Advanced tab.

The best way to find any config is to lookup in the search/filter bar on the top/right as shown below.

When you search for "yarn.scheduler.minimum-allocation-mb" in that search box it will show you only the "Minimum Container Size (Memory)" slider as shown in the first image above.

Re: Overwrite yarn, mapred, tez and hive configurations after restarting services in Ambari

Sanaz_janbakhsh — Wed, 06 Sep 2017 05:16:53 GMT

Hi,

I have two issues :

1- I can see this parameter but it doesn't let me to increase that much (server has 250G memory but it let me to increase it up to 3976MB. untitled1.png

2- when i run calculator, it recomends to change serveral parameters per below. I cannot find them in the GUI.

***** mapred-site.xml *****

mapreduce.map.memory.mb=32768

mapreduce.map.java.opts=-Xmx25600m

mapreduce.reduce.memory.mb=32768

mapreduce.reduce.java.opts=-Xmx25600m

mapreduce.task.io.sort.mb=1792

***** yarn-site.xml *****

yarn.scheduler.minimum-allocation-mb=32768

yarn.scheduler.maximum-allocation-mb=262144

yarn.nodemanager.resource.memory-mb=262144

yarn.app.mapreduce.am.resource.mb=32768

yarn.app.mapreduce.am.command-opts=-Xmx25600m

***** tez-site.xml *****

tez.am.resource.memory.mb=32768

tez.am.java.opts=-Xmx25600m

***** hive-site.xml *****

hive.tez.container.size=32768

hive.tez.java.opts=-Xmx25600m

hive.auto.convert.join.noconditionaltask.size=8589934000

Please advise.

Re: Overwrite yarn, mapred, tez and hive configurations after restarting services in Ambari

gsaha — Thu, 07 Sep 2017 01:21:29 GMT

1- You can always click on the red-pen edit button and manually enter the exact value you want.

2- You can uncheck and ignore the changes recommended by Ambari or accept the changes. To view mapred, tez and hive config properties, make sure you are in the corresponding services tab.