Posts: 32
Registered: ‎09-25-2016
Accepted Solution

yarn logs location on disk

Is there a YARN API or command to know path to yarn logs location on disk for given container and application id ? Also want to add; we don't have log aggregation working and I'm perticularly looking for direct physical link to the file not the web interface.




Posts: 177
Topics: 8
Kudos: 27
Solutions: 19
Registered: ‎07-16-2015

Re: yarn logs location on disk

Not sure this information is available.


You could go with the "yarn logs" command

or go with the basic way using command line :

- pdsh to distribute the same command on every data-node

- launch a find on the container id








Posts: 32
Registered: ‎09-25-2016

Re: yarn logs location on disk

Finding logs manually in machine sound very brute force; I was thinking more of an API or CLI option to find logs


Anyway the main issue we're trying to solve is access to logs to all developers in prod environment. Our node managers are behind the bars and not accessible ( any port or web ) to develoeprs and it's unlikely to happen. So we're trying to find a way to proxy the logs.


I discovered that there is a jobhistory proxy to look at completed jobs / yarn apps but I coudln't get it working for running app. Is there any trick / way to access running app's logs like above ?