Change Yarn config on Cloudera Manager Web UI doesn't affect yarn-site.xml file at all?

quangbilly79 — Thu, 08 Jun 2023 05:00:21 GMT

I'm using CDH 6.2. I add and change some Yarn config inside Cloudera Manager Web UI (related to yarn shuffle service), something like "NodeManager Advanced Configuration Snippet (Safety Valve) for yarn-site.xml"

I preview the result and Cloudera shows that some lines will be added to yarn-site.xml. But after restarting everything, I check yarn-site.xml at "/etc/hadoop/conf.cloudera.yarn/yarn-site.xml", I can't find any setting above, even the word "shuffle" doesn't appear anywhere at all? I also check for other files like marped-site.xml or core-site.xml, no luck so far

So where the heck do all the settings above add to? I'm confused. I need the proper yarn-site.xml file with all the settings just like in the Cloudera web UI

Re: Change Yarn config on Cloudera Manager Web UI doesn't affect yarn-site.xml file at all?

Rajat_710 — Mon, 22 Apr 2024 07:32:19 GMT

Hi @quangbilly79 any advanced config will be treated as CM advanced config, CM is overriding it, so I tried the same config in my test cluster and it will be stored in the process directory yarn-site.xml file.

You can ssh to your Yarn NM node and check the latest yarn process directory for the same and find the shuffle property in it.

Example path:

/var/run/cloudera-scm-agent/process/415-yarn-NODEMANAGER/yarn-site.xml

question Re: Change Yarn config on Cloudera Manager Web UI doesn't affect yarn-site.xml file at all? in Support Questions

Change Yarn config on Cloudera Manager Web UI doesn't affect yarn-site.xml file at all?

Re: Change Yarn config on Cloudera Manager Web UI doesn't affect yarn-site.xml file at all?