Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How does Application master knows resource name (node name/rack name).?

Highlighted

How does Application master knows resource name (node name/rack name).?

Explorer

As per YARN specifications, A resource request from Application master will have following properties
1) Desired Resource Name (host name, rack name or * for no preference)
2) Priority
3) magnitude/requirements
4) No of containers

 

My question here is, how does AM knows the Rack or Node name even before allocating the container.?
Will it first reach Resource Manager and gets the Node’s/Block’s information .? and then requests for container to be launched in that node.?

 

 

1 REPLY 1

Re: How does Application master knows resource name (node name/rack name).?

Master Guru
Yes, you could gather this info at the client-side before the AM is triggered. That is how we do it in MR2, for example.

The InputSplit created at the client-end [1] carry the HDFS-provided info about the host and rack, which we end up using later in the submitted MR AM to schedule the actual containers [2].

[1] - https://github.com/cloudera/hadoop-common/blob/cdh5.4.0-release/hadoop-mapreduce-project/hadoop-mapr... and https://github.com/cloudera/hadoop-common/blob/cdh5.4.0-release/hadoop-mapreduce1-project/src/mapred...
[2] - https://github.com/cloudera/hadoop-common/blob/cdh5.4.0-release/hadoop-mapreduce-project/hadoop-mapr... and https://github.com/cloudera/hadoop-common/blob/cdh5.4.0-release/hadoop-mapreduce-project/hadoop-mapr...
Don't have an account?
Coming from Hortonworks? Activate your account here