Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to fetch the number of containers and Vcores allocated for a application once it is completed from YARN/ATS REST API?

Highlighted

How to fetch the number of containers and Vcores allocated for a application once it is completed from YARN/ATS REST API?

Expert Contributor

I'm following YARN REST API this shows:

allocatedMBintThe sum of memory in MB allocated to the application’s running containers
allocatedVCoresintThe sum of virtual cores allocated to the application’s running containers

But these are the aggregated metrics.

I'm looking for the total containers, and for each container how much memory and vcores are allocated.

Is there a way this can be achieved?

Thanks

Venkat

2 REPLIES 2

Re: How to fetch the number of containers and Vcores allocated for a application once it is completed from YARN/ATS REST API?

Hi @Venkata Sudheer Kumar M,

You can fetch total containers allocated for an application using YARN CLI -

1. Fetch applicationAttempt for the application using-  yarn applicationattempt -list <applicationID>
2. Fetch all containers for that applicationAttempt using-  yarn  container -list <appattempt>

In the step 2 above, you will also get NodeManager info on which the container was launched. From the nodemanager info on which the container was launched, you can obtain the vcores and memory allocated for that container using the below REST API

curl http://<Nodemanager address>:<port>/ws/v1/node/containers/<containerID>

Hope this is helpful to you!

Re: How to fetch the number of containers and Vcores allocated for a application once it is completed from YARN/ATS REST API?

Expert Contributor
@ssathish

Isn't it only for the currently running jobs? Do we able to see the job completed jobs containers and details.

Here is the Running job that shows Total Allocated Containers:running-containers.png

Here is the Completed Job that shows Total Allocated Containers: finished-job.png

But none of these Total Allocated Containers the get transformed to the REST API of RM. Below given XML's will show only the allocated contain

Running Job XML: running.xml

Finished Job XML: finished-job.xml

And the Node REST API:

curl http://<Nodemanager address>:<port>/ws/v1/node/containers/<containerID>

gives the containers details about only the running containers not about the completed containers.

Is there a way what we see on YARN Application UI https://manag003:8090/cluster/appattempt/appattempt_1522212350151_40488_000001 for the Total Allocated Containers:

to be transformed to REST API.

Thanks

Venkat