Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Getting a physical machines with all its cores from YARN

avatar
New Contributor

Is it possible to ask YARN to allocate one full physical machine (with all of its cores) to one application? E.g. If I need 24 cores and there is a machine in the cluster which has 24 cores, I would like to get that full physical box instead of 24 vcores which are potentially scattered in the cluster. Or if I have apriori knowledge that in my cluster no machine has less than n cores, I would like YARN to give me x machines (with all its cores) instead of asking in terms of vcores.

1 ACCEPTED SOLUTION

avatar
Master Guru
@Abdul Qadeer

I see what are you asking. Basically you want your application to be run on dedicated nodemanager. You can have a look at YARN's node label feature.

https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeLabel.html

Please do let me know if this is not what you are looking for.

View solution in original post

3 REPLIES 3

avatar
Master Guru
@Abdul Qadeer

I see what are you asking. Basically you want your application to be run on dedicated nodemanager. You can have a look at YARN's node label feature.

https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeLabel.html

Please do let me know if this is not what you are looking for.

avatar
New Contributor

@Kuldeep Kulkarni

Thank you for giving me a good starting point. Let me do some experimentation using YARN's node labels and then I might have some follow-up questions.

avatar
Master Guru

@Abdul Qadeer

You are welcome Sir! If this helped you, you can accept my answer and post a new question whenever you need our help.