Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to fetch the number of containers and Vcores allocated for a application once it is completed from YARN/ATS REST API?

avatar
Expert Contributor

I'm following YARN REST API this shows:

allocatedMBintThe sum of memory in MB allocated to the application’s running containers
allocatedVCoresintThe sum of virtual cores allocated to the application’s running containers

But these are the aggregated metrics.

I'm looking for the total containers, and for each container how much memory and vcores are allocated.

Is there a way this can be achieved?

Thanks

Venkat

2 REPLIES 2

avatar

Hi @Venkata Sudheer Kumar M,

You can fetch total containers allocated for an application using YARN CLI -

1. Fetch applicationAttempt for the application using-  yarn applicationattempt -list <applicationID>
2. Fetch all containers for that applicationAttempt using-  yarn  container -list <appattempt>

In the step 2 above, you will also get NodeManager info on which the container was launched. From the nodemanager info on which the container was launched, you can obtain the vcores and memory allocated for that container using the below REST API

curl http://<Nodemanager address>:<port>/ws/v1/node/containers/<containerID>

Hope this is helpful to you!

avatar
Expert Contributor
@ssathish

Isn't it only for the currently running jobs? Do we able to see the job completed jobs containers and details.

Here is the Running job that shows Total Allocated Containers:running-containers.png

Here is the Completed Job that shows Total Allocated Containers: finished-job.png

But none of these Total Allocated Containers the get transformed to the REST API of RM. Below given XML's will show only the allocated contain

Running Job XML: running.xml

Finished Job XML: finished-job.xml

And the Node REST API:

curl http://<Nodemanager address>:<port>/ws/v1/node/containers/<containerID>

gives the containers details about only the running containers not about the completed containers.

Is there a way what we see on YARN Application UI https://manag003:8090/cluster/appattempt/appattempt_1522212350151_40488_000001 for the Total Allocated Containers:

to be transformed to REST API.

Thanks

Venkat