https://community.cloudera.com/t5/Support-Questions/How-to-make-Yarn-deploy-resources-to-new-added-node/m-p/377279#M243204 <P>When we submit the spark using YARN, based on YARN resources application will run. In your case you need to add more YARN Gateway nodes to process with more resources. We can't process the data by only added new nodes and yarn will distribute processing all nodes.</P> Thu, 05 Oct 2023 09:09:09 GMT RangaReddy 2023-10-05T09:09:09Z https://community.cloudera.com/t5/Support-Questions/How-to-make-Yarn-deploy-resources-to-new-added-node/m-p/374387#M241969 <P>I added some new nodes to my cluster and it works fine. Then I add Spark Gateway roles to all the new nodes. We're using Yarn to manage and distribute Spark work.</P><P>Does adding&nbsp;Spark Gateway roles to new nodes enough to make Yarn think like "Hey there are some new nodes here, let's distribute some containers and work to these new nodes"? Or do I have to add Yarn Gateway roles to these new nodes too?</P><P>How to make sure that Yarn will use these new nodes when executing jobs to reduce the overall workload of my cluster</P><P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="quangbilly79_0-1690257406237.png" style="width: 400px;"><img src="https://community.cloudera.com/t5/image/serverpage/image-id/38077i9546C33308EBB36E/image-size/medium?v=v2&amp;px=400" role="button" title="quangbilly79_0-1690257406237.png" alt="quangbilly79_0-1690257406237.png" /></span></P><P>&nbsp;</P><P>&nbsp;</P> Tue, 25 Jul 2023 03:58:32 GMT https://community.cloudera.com/t5/Support-Questions/How-to-make-Yarn-deploy-resources-to-new-added-node/m-p/374387#M241969 quangbilly79 2023-07-25T03:58:32Z https://community.cloudera.com/t5/Support-Questions/How-to-make-Yarn-deploy-resources-to-new-added-node/m-p/377279#M243204 <P>When we submit the spark using YARN, based on YARN resources application will run. In your case you need to add more YARN Gateway nodes to process with more resources. We can't process the data by only added new nodes and yarn will distribute processing all nodes.</P> Thu, 05 Oct 2023 09:09:09 GMT https://community.cloudera.com/t5/Support-Questions/How-to-make-Yarn-deploy-resources-to-new-added-node/m-p/377279#M243204 RangaReddy 2023-10-05T09:09:09Z https://community.cloudera.com/t5/Support-Questions/How-to-make-Yarn-deploy-resources-to-new-added-node/m-p/377928#M243379 <P>The node must have a NodeManager role to take part of the processing, Spark gateway, and Yarn Gateway</P> Thu, 19 Oct 2023 23:36:41 GMT https://community.cloudera.com/t5/Support-Questions/How-to-make-Yarn-deploy-resources-to-new-added-node/m-p/377928#M243379 Fawze 2023-10-19T23:36:41Z