Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Change Yarn config on Cloudera Manager Web UI doesn't affect yarn-site.xml file at all?

avatar
Contributor

I'm using CDH 6.2. I add and change some Yarn config inside Cloudera Manager Web UI (related to yarn shuffle service), something like "NodeManager Advanced Configuration Snippet (Safety Valve) for yarn-site.xml"

quangbilly79_0-1686199492751.png

I preview the result and Cloudera shows that some lines will be added to yarn-site.xml. But after restarting everything, I check yarn-site.xml at "/etc/hadoop/conf.cloudera.yarn/yarn-site.xml", I can't find any setting above, even the word "shuffle" doesn't appear anywhere at all? I also check for other files like marped-site.xml or core-site.xml, no luck so far

quangbilly79_1-1686200195481.png

 

 

So where the heck do all the settings above add to? I'm confused. I need the proper yarn-site.xml file with all the settings just like in the Cloudera web UI

 

 

1 REPLY 1

avatar
Expert Contributor

Hi @quangbilly79 any advanced config will be treated as CM advanced config, CM is overriding it, so I tried the same config in my test cluster and it will be stored in the process directory yarn-site.xml file.

You can ssh to your Yarn NM node and check the latest yarn process directory for the same and find the shuffle property in it.

Example path: 

/var/run/cloudera-scm-agent/process/415-yarn-NODEMANAGER/yarn-site.xml