env: hdp 220.127.116.11 there are 5 nodes. I want to use 2 high spec machine to run spark job, and all nodes can run mapreduce job.
My design is use a nodelabel with non-exclusive: yarn rmadmin -addToClusterNodeLabels "high(exclusive=false)" yarn rmadmin -replaceLabelsOnNode "node4=high node5=high"
And set two queues, one called mr, and another one called spark.
When I submit job to mr, and high nodelabel nodes are idle, Can my job use all the rescoure?
My test result is something different. When I run a heavy job, the 3 nodes were used fast and the two high labeled nodes, just started one or two container(1%), I confirmed the capacity-scheduler, it can use more than 50%.
I am not sure how to config the shareable function now...