Reply
Highlighted
New Contributor
Posts: 4
Registered: ‎01-25-2014

Yarn logs to hdfs

Hi!

 

I am wondering.. can you store all yarn logs to hdfs directly and not to a local server folder?

Explorer
Posts: 22
Registered: ‎01-08-2016

Re: Yarn logs to hdfs

yarn logs will be stored in local not hdfs

Posts: 1,903
Kudos: 435
Solutions: 305
Registered: ‎07-31-2013

Re: Yarn logs to hdfs

With log aggregation turned on, logs from containers in YARN typically get uploaded to HDFS at the end of the container's life, and then deleted from the local disks.

Could you expand on what lead you to this question/what you are observing and are looking to change?