Support Questions

Find answers, ask questions, and share your expertise

Change Yarn config on Cloudera Manager Web UI doesn't affect yarn-site.xml file at all?

avatar
Contributor

I'm using CDH 6.2. I add and change some Yarn config inside Cloudera Manager Web UI (related to yarn shuffle service), something like "NodeManager Advanced Configuration Snippet (Safety Valve) for yarn-site.xml"

quangbilly79_0-1686199492751.png

I preview the result and Cloudera shows that some lines will be added to yarn-site.xml. But after restarting everything, I check yarn-site.xml at "/etc/hadoop/conf.cloudera.yarn/yarn-site.xml", I can't find any setting above, even the word "shuffle" doesn't appear anywhere at all? I also check for other files like marped-site.xml or core-site.xml, no luck so far

quangbilly79_1-1686200195481.png

 

 

So where the heck do all the settings above add to? I'm confused. I need the proper yarn-site.xml file with all the settings just like in the Cloudera web UI

 

 

1 REPLY 1

avatar
Super Collaborator

Hi @quangbilly79 any advanced config will be treated as CM advanced config, CM is overriding it, so I tried the same config in my test cluster and it will be stored in the process directory yarn-site.xml file.

You can ssh to your Yarn NM node and check the latest yarn process directory for the same and find the shuffle property in it.

Example path: 

/var/run/cloudera-scm-agent/process/415-yarn-NODEMANAGER/yarn-site.xml