Created 10-13-2020 07:23 AM
Hello.
In my cloudera cluser i'm having constantly working script(hes never stop) and logs in local dir's always grows ( local folder /yarn/nm/).
[root@sdl-masternode ~]# du -s -h /yarn/nm/usercache/azat/appcache/*
658M /yarn/nm/usercache/azat/appcache/application_1599754680433_0192
2.1G /yarn/nm/usercache/azat/appcache/application_1599754680433_0203
12K /yarn/nm/usercache/azat/appcache/application_1599754680433_0210
6.7G /yarn/nm/usercache/azat/appcache/application_1599754680433_0241
605M /yarn/nm/usercache/azat/appcache/application_1599754680433_0254
40K /yarn/nm/usercache/azat/appcache/application_1600498608856_0120
2.3G /yarn/nm/usercache/azat/appcache/application_1600498608856_0150
12G /yarn/nm/usercache/azat/appcache/application_1600498608856_0155
1.2G /yarn/nm/usercache/azat/appcache/application_1600498608856_0381
4.6G /yarn/nm/usercache/azat/appcache/application_1600498608856_0386
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0398
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0400
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0613
46M /yarn/nm/usercache/azat/appcache/application_1600498608856_0688
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0966
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0974
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0983
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0986
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0989
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0991
16K /yarn/nm/usercache/azat/appcache/application_1600498608856_1002
96K /yarn/nm/usercache/azat/appcache/application_1600498608856_1003
How can i automatically clean it, for example, if log older then 7 days - its must be retain from local.
Created 10-20-2020 12:07 PM
@Yuriy_but For CDH related service like Yarn you can configure NodeManager Maximum Log File Backups in Service Configuration but if these are some custom logs then I would say setup a cron job to delete these periodically.
Created 10-23-2020 02:35 AM
Yes, i thought about cron. But maybe smth in yarn conf can help, but i miss it. Maximum log files bk doesnt help