Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

Change Yarn config on Cloudera Manager Web UI doesn't affect yarn-site.xml file at all?

avatar
Contributor

I'm using CDH 6.2. I add and change some Yarn config inside Cloudera Manager Web UI (related to yarn shuffle service), something like "NodeManager Advanced Configuration Snippet (Safety Valve) for yarn-site.xml"

quangbilly79_0-1686199492751.png

I preview the result and Cloudera shows that some lines will be added to yarn-site.xml. But after restarting everything, I check yarn-site.xml at "/etc/hadoop/conf.cloudera.yarn/yarn-site.xml", I can't find any setting above, even the word "shuffle" doesn't appear anywhere at all? I also check for other files like marped-site.xml or core-site.xml, no luck so far

quangbilly79_1-1686200195481.png

 

 

So where the heck do all the settings above add to? I'm confused. I need the proper yarn-site.xml file with all the settings just like in the Cloudera web UI

 

 

1 REPLY 1

avatar
Super Collaborator

Hi @quangbilly79 any advanced config will be treated as CM advanced config, CM is overriding it, so I tried the same config in my test cluster and it will be stored in the process directory yarn-site.xml file.

You can ssh to your Yarn NM node and check the latest yarn process directory for the same and find the shuffle property in it.

Example path: 

/var/run/cloudera-scm-agent/process/415-yarn-NODEMANAGER/yarn-site.xml