How to fetch the number of containers and Vcores allocated for a application once it is completed from YARN/ATS REST API?

[#1]

I'm following YARN REST API this shows:

allocatedMB	int	The sum of memory in MB allocated to the application’s running containers
allocatedVCores	int	The sum of virtual cores allocated to the application’s running containers

But these are the aggregated metrics.

I'm looking for the total containers, and for each container how much memory and vcores are allocated.

Is there a way this can be achieved?

Thanks

Venkat

[#2]

Hi @Venkata Sudheer Kumar M,

You can fetch total containers allocated for an application using YARN CLI -

1. Fetch applicationAttempt for the application using-  yarn applicationattempt -list <applicationID>
2. Fetch all containers for that applicationAttempt using-  yarn  container -list <appattempt>

In the step 2 above, you will also get NodeManager info on which the container was launched. From the nodemanager info on which the container was launched, you can obtain the vcores and memory allocated for that container using the below REST API

curl http://<Nodemanager address>:<port>/ws/v1/node/containers/<containerID>

Hope this is helpful to you!

[#3]

@ssathish

Isn't it only for the currently running jobs? Do we able to see the job completed jobs containers and details.

Here is the Running job that shows Total Allocated Containers:running-containers.png

Here is the Completed Job that shows Total Allocated Containers: finished-job.png

But none of these Total Allocated Containers the get transformed to the REST API of RM. Below given XML's will show only the allocated contain

Running Job XML: running.xml

Finished Job XML: finished-job.xml

And the Node REST API:

curl http://<Nodemanager address>:<port>/ws/v1/node/containers/<containerID>

gives the containers details about only the running containers not about the completed containers.

Is there a way what we see on YARN Application UI https://manag003:8090/cluster/appattempt/appattempt_1522212350151_40488_000001 for the Total Allocated Containers:

to be transformed to REST API.

Thanks

Venkat