Resource Manager API's

[#1]

Looking at hitting allt he right API's for charge back scenario. In the following call to RM API's

http://192.168.245.131:8088/ws/v1/cluster/apps/application_1442459299785_0002

1. What is memory Seconds, how to measure the memory used by an application using this number
2. What is vcoreSEconds ? Is it directly proportional to the CPU seconds ?
3. Is running containers supposed to show us all the containers launched for that app ?
4. Allocated MB, allocated VCores are also showing invalid values or zeros.
5. How frequently can I call this API. I don’t want to miss any records, but at the same time we don’t want duplicate records.

Output of call below

<app>
<id>application_1442459299785_0002</id>
<user>hive</user>
<name>HIVE-521c0020-4786-4e55-8822-abe11a517ba1</name>
<queue>default</queue>
<state>FINISHED</state>
<finalStatus>SUCCEEDED</finalStatus>
<progress>100.0</progress>
<trackingUI>History</trackingUI><trackingUrl>
http://sandbox.hortonworks.com:8088/proxy/application_1442459299785_0002/
</trackingUrl>
<diagnostics>null</diagnostics>
<clusterId>1444871487358</clusterId>
<applicationType>TEZ</applicationType>
<applicationTags/>
<startedTime>1442460228059</startedTime>
<finishedTime>1442502311402</finishedTime>
<elapsedTime>42083343</elapsedTime>
<amContainerLogs>
http://sandbox.hortonworks.com:8042/node/containerlogs/container_e17_1442459299785_0002_01_000001/hi...
</amContainerLogs>
<amHostHttpAddress>sandbox.hortonworks.com:8042</amHostHttpAddress>
<allocatedMB>-1</allocatedMB>
<allocatedVCores>-1</allocatedVCores>
<runningContainers>-1</runningContainers>
<memorySeconds>10540022</memorySeconds>
<vcoreSeconds>42158</vcoreSeconds>
<preemptedResourceMB>0</preemptedResourceMB>
<preemptedResourceVCores>0</preemptedResourceVCores>
<numNonAMContainerPreempted>0</numNonAMContainerPreempted>
<numAMContainerPreempted>0</numAMContainerPreempted>
<logAggregationStatus>NOT_START</logAggregationStatus>
</app>

[#2]

Some information in this link is useful looking under this section

Elements of the app (Application) object

, but seems a little vague, with what the meanings are to the above output. Maybe the -1's are due to the application being completed already? Curious to get others thoughts on how or if they have done a similar scenario for charge back purposes before.

[#3]

Looks like it is because of the application being completed because when running there is much more detail in this output of this call. I think I'm ok at this point, but please share experiences

[#4 · Accepted solution]

Got this information from Sai Nukavarapu and wanted to add it back to the thread in case anyone else needs to have more context to these meanings.

- What MemorySeconds means and vCoreSeconds mean.

1. memorySeconds - The amount of memory the application has allocated (megabyte-seconds);

Aggregated amount of memory (in megabytes) the application has allocated X the number of seconds the application has been running.

2. The amount of CPU resources the application has allocated (virtual core-seconds)

Aggregated number of vcores that the application has allocated X the number of seconds the application has been running.