Support Questions
Find answers, ask questions, and share your expertise

Getting a physical machines with all its cores from YARN

New Contributor

Is it possible to ask YARN to allocate one full physical machine (with all of its cores) to one application? E.g. If I need 24 cores and there is a machine in the cluster which has 24 cores, I would like to get that full physical box instead of 24 vcores which are potentially scattered in the cluster. Or if I have apriori knowledge that in my cluster no machine has less than n cores, I would like YARN to give me x machines (with all its cores) instead of asking in terms of vcores.

1 ACCEPTED SOLUTION

Super Guru
@Abdul Qadeer

I see what are you asking. Basically you want your application to be run on dedicated nodemanager. You can have a look at YARN's node label feature.

https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeLabel.html

Please do let me know if this is not what you are looking for.

View solution in original post

3 REPLIES 3

Super Guru
@Abdul Qadeer

I see what are you asking. Basically you want your application to be run on dedicated nodemanager. You can have a look at YARN's node label feature.

https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeLabel.html

Please do let me know if this is not what you are looking for.

New Contributor

@Kuldeep Kulkarni

Thank you for giving me a good starting point. Let me do some experimentation using YARN's node labels and then I might have some follow-up questions.

Super Guru

@Abdul Qadeer

You are welcome Sir! If this helped you, you can accept my answer and post a new question whenever you need our help.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.