question Resource Manager API's in Archives of Support Questions (Read Only)

Resource Manager API's

drice1 — Fri, 16 Sep 2022 09:44:16 GMT

Looking at hitting allt he right API's for charge back scenario. In the following call to RM API's

http://192.168.245.131:8088/ws/v1/cluster/apps/application_1442459299785_0002

What is memory Seconds, how to measure the memory used by an application using this number
What is vcoreSEconds ? Is it directly proportional to the CPU seconds ?
Is running containers supposed to show us all the containers launched for that app ?
Allocated MB, allocated VCores are also showing invalid values or zeros.
How frequently can I call this API. I don’t want to miss any records, but at the same time we don’t want duplicate records.

Output of call below

<app>
<id>application_1442459299785_0002</id>
<user>hive</user>
<name>HIVE-521c0020-4786-4e55-8822-abe11a517ba1</name>
<queue>default</queue>
<state>FINISHED</state>
<finalStatus>SUCCEEDED</finalStatus>
<progress>100.0</progress>
<trackingUI>History</trackingUI><trackingUrl>
http://sandbox.hortonworks.com:8088/proxy/application_1442459299785_0002/
</trackingUrl>
<diagnostics>null</diagnostics>
<clusterId>1444871487358</clusterId>
<applicationType>TEZ</applicationType>
<applicationTags/>
<startedTime>1442460228059</startedTime>
<finishedTime>1442502311402</finishedTime>
<elapsedTime>42083343</elapsedTime>
<amContainerLogs>
http://sandbox.hortonworks.com:8042/node/containerlogs/container_e17_1442459299785_0002_01_000001/hive
</amContainerLogs>
<amHostHttpAddress>sandbox.hortonworks.com:8042</amHostHttpAddress>
<allocatedMB>-1</allocatedMB>
<allocatedVCores>-1</allocatedVCores>
<runningContainers>-1</runningContainers>
<memorySeconds>10540022</memorySeconds>
<vcoreSeconds>42158</vcoreSeconds>
<preemptedResourceMB>0</preemptedResourceMB>
<preemptedResourceVCores>0</preemptedResourceVCores>
<numNonAMContainerPreempted>0</numNonAMContainerPreempted>
<numAMContainerPreempted>0</numAMContainerPreempted>
<logAggregationStatus>NOT_START</logAggregationStatus>
</app>

Re: Resource Manager API's

drice1 — Thu, 15 Oct 2015 20:57:25 GMT

Some information in this link is useful looking under this section

Elements of the app (Application) object

, but seems a little vague, with what the meanings are to the above output. Maybe the -1's are due to the application being completed already? Curious to get others thoughts on how or if they have done a similar scenario for charge back purposes before.

Re: Resource Manager API's

drice1 — Thu, 15 Oct 2015 21:08:04 GMT

Looks like it is because of the application being completed because when running there is much more detail in this output of this call. I think I'm ok at this point, but please share experiences

Re: Resource Manager API's

drice1 — Tue, 27 Oct 2015 00:59:04 GMT

Got this information from Sai Nukavarapu and wanted to add it back to the thread in case anyone else needs to have more context to these meanings.

- What MemorySeconds means and vCoreSeconds mean.

1. memorySeconds - The amount of memory the application has allocated (megabyte-seconds);

Aggregated amount of memory (in megabytes) the application has allocated X the number of seconds the application has been running.

2. The amount of CPU resources the application has allocated (virtual core-seconds)

Aggregated number of vcores that the application has allocated X the number of seconds the application has been running.