Removing YARN job summary

[#1]

Does anyone know how to remove old jobs from the resource manager web UI: http://<resourcemanager:8088/cluster I assume they are held on the timeline server but they don't seem to ever be removed. The jobs listed there are from February when we began testing the cluster and viewing the jobs in Apache Hue produces an error saying they are expired. Is there anyway to clear the summary completely?

[#2 · Accepted solution]

Set the retention and aggregations timeline for YARN logging. YARN logs, including summary will be removed after the set timeline.

Go to https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-common/yarn-default.xml and search for "log" and you will find the parameters that can allow you manage log retention, whether is detailed log or aggregated.

I think you are looking for

yarn.resourcemanager.max-completed-applications, which is set to 10,000 by default. You could change that value via Ambari. You may have to restart Ambari. Test in a development or test environment to determine the best combination of configurations for your case.

A good blog, even a bit old:

http://hortonworks.com/blog/simplifying-user-logs-management-and-access-in-yarn/

[#3]

Many thanks! I've managed to clear the job history.

[#4]

Very elaborate answer!

[#5]

@Constantin Stanca @Dinesh Chitlangia

Hello,

I'm facing the same issue and i've explained it through this question:

https://community.hortonworks.com/questions/148642/how-can-i-remove-yarn-pplications-from-ambari-ser....

I've set "yarn.resourcemanager.max-completed-applications" to 10 and i've restarted my ambari-server. I got the same total of yarn applications through RM UI. Is there any step missed ?

I'll be grateful if you could help me fixing this issue.

[#6]

Restart your RM