yarn logs location on disk

[#1]

Is there a YARN API or command to know path to yarn logs location on disk for given container and application id ? Also want to add; we don't have log aggregation working and I'm perticularly looking for direct physical link to the file not the web interface.

Thanks,

Sunil

[#2 · Accepted solution]

Not sure this information is available.

You could go with the "yarn logs" command

or go with the basic way using command line :

- pdsh to distribute the same command on every data-node

- launch a find on the container id

regards,

mathieu

[#3]

Finding logs manually in machine sound very brute force; I was thinking more of an API or CLI option to find logs

Anyway the main issue we're trying to solve is access to logs to all developers in prod environment. Our node managers are behind the bars and not accessible ( any port or web ) to develoeprs and it's unlikely to happen. So we're trying to find a way to proxy the logs.

I discovered that there is a jobhistory proxy to look at completed jobs / yarn apps but I coudln't get it working for running app. Is there any trick / way to access running app's logs like above ?

http://resourcemanager.xyz.com:19888/jobhistory/logs//dataNode.com:8041/container_id_000001/containe...