question Re: Removing YARN job summary in Support Questions

Removing YARN job summary

alan9270 — Fri, 09 Sep 2016 21:04:42 GMT

Does anyone know how to remove old jobs from the resource manager web UI: http://<resourcemanager:8088/cluster I assume they are held on the timeline server but they don't seem to ever be removed. The jobs listed there are from February when we began testing the cluster and viewing the jobs in Apache Hue produces an error saying they are expired. Is there anyway to clear the summary completely?

Re: Removing YARN job summary

cstanca — Thu, 29 Dec 2016 06:07:17 GMT

Set the retention and aggregations timeline for YARN logging. YARN logs, including summary will be removed after the set timeline.

Go to https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-common/yarn-default.xml and search for "log" and you will find the parameters that can allow you manage log retention, whether is detailed log or aggregated.

I think you are looking for

yarn.resourcemanager.max-completed-applications, which is set to 10,000 by default. You could change that value via Ambari. You may have to restart Ambari. Test in a development or test environment to determine the best combination of configurations for your case.

A good blog, even a bit old:

http://hortonworks.com/blog/simplifying-user-logs-management-and-access-in-yarn/

Re: Removing YARN job summary

alan9270 — Mon, 09 Jan 2017 19:22:38 GMT

Many thanks! I've managed to clear the job history.

Re: Removing YARN job summary

dineshc — Tue, 10 Jan 2017 14:09:46 GMT

Very elaborate answer!

Re: Removing YARN job summary

kabadou_rawia — Wed, 29 Nov 2017 17:23:31 GMT

@Constantin Stanca @Dinesh Chitlangia

Hello,

I'm facing the same issue and i've explained it through this question:

https://community.hortonworks.com/questions/148642/how-can-i-remove-yarn-pplications-from-ambari-serv.html.

I've set "yarn.resourcemanager.max-completed-applications" to 10 and i've restarted my ambari-server. I got the same total of yarn applications through RM UI. Is there any step missed ?

I'll be grateful if you could help me fixing this issue.

Re: Removing YARN job summary

raravena80 — Thu, 21 Dec 2017 06:26:09 GMT

Restart your RM