Browsing logs of running YARN app from yarn log CLI

[#1]

Hi,

AFAIK `yarn logs` command could be used to view aggregated logs of finsihed YARN applications. For ones that not finished yet, you had to either use YARN UI or ssh to node managers.

However on Hortonworks page I see their yarn logs works for running apps as well already: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_yarn-resource-management/content/ch_yarn...

Is there any plan/way to make it work for cloudera as well?

Thanks!

[#2]

What they have done is turn on the partial aggregation via the setting: yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds.

That will allow you to grab some of the logs using the command line.

We do not support this in CDH although we have the exact same code in CDH is available upstream. We have tested the setting and found that it breaks log access via the different UIs in multiple ways. So you get a working command line in 99% of the cases but when you try to use the RM or AM UIs it breaks almost always. The way it breaks changes over time for the same application. That is not a feature that we can support in the state it is at the moment.

Wilfred