question Re: What is the difference between timeline server api and resource manager api calls for applications in Support Questions

What is the difference between timeline server api and resource manager api calls for applications

vishalprakash_s — Tue, 20 Dec 2016 18:15:10 GMT

1) hostname:8188/ws/v1/applicationhistor/apps

2) hostname:8088/ws/vi/cluster/apps

The first url is rest api call from timeline server and second url is from RM api. Both the url's are supposed to return list of ALL applications (succeeded/killed/failed) run on the cluster.

But there are a number of applications appearing in timeline server api call but NOT in RM api call.

Is this difference expected? What is the purpose/difference between these 2 approach of getting list of application run on cluster?

For example:

application_1477690913969_0092 and application_1477690913969_0093 appears in timeline svr api call but doesn't gets listed in RM api call.

Please see the attached screenshots.screen2.jpg screen3.jpg

Re: What is the difference between timeline server api and resource manager api calls for applications

KuldeepK — Tue, 20 Dec 2016 23:58:19 GMT

@Vishal Prakash Shah

I'm not 100% sure but I think this is expected as RM does not keep historical information of all the Applications. Main purpose of Yarn Application Time server is to maintain historical information(yarn.timeline-service.ttl-ms is the parameter for retention) about all the YARN jobs hence you see lot of results with Timeline API.

Default value of yarn.timeline-service.ttl-ms is 2678400000 ms i.e. 31 days

You can read more about timeline server here - https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/TimelineServer.html

Hope this information helps!

Re: What is the difference between timeline server api and resource manager api calls for applications

vishalprakash_s — Wed, 21 Dec 2016 00:06:39 GMT

In my setup, yarn.timeline-service.ttl-ms is configured with default value. Still I can see the both api's returning application details for more than 3 months. Any idea why??

Re: What is the difference between timeline server api and resource manager api calls for applications

KuldeepK — Wed, 21 Dec 2016 00:14:52 GMT

@Vishal Prakash Shah - Can you please post value of your yarn.timeline-service.ttl-ms?

Re: What is the difference between timeline server api and resource manager api calls for applications

vishalprakash_s — Wed, 21 Dec 2016 00:16:33 GMT

My question is-

if application_1477690913969_0092 is shown as an application being run on cluster, then why command yarn application -status application_1477690913969_0092 is giving error. The application was run on 30th Nov, 2016. It appears in timeline server api call but not in resource manager api call. There are results in rm api call which were run before 30th Nov.

Re: What is the difference between timeline server api and resource manager api calls for applications

vishalprakash_s — Wed, 21 Dec 2016 00:18:54 GMT

It is 2678400000 ttl.jpg