Support Questions

Find answers, ask questions, and share your expertise

Yarn Log Aggregation Directory

avatar
Contributor

Hi all,
I would like to know if it is possible to specify a location other than HDFS (eg. remote NFS Storage) for the parameter yarn_nodemanager_remote_app_log_dir because we don't have enough space on HDFS and we need to maintain the current policy retention (7 days).
Thanks in advance.

2 REPLIES 2

avatar
Expert Contributor

Yes you can use but the problem with that will be you have to allow all the users to access that directory. YARN can move local logs securely onto HDFS or a cloud-based storage, such as AWS. This allows the logs to be stored for a much longer time than they could be on a local disk, allows faster search for a particular log file and optionally can handle compression.

avatar
Community Manager

@Ivoz Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community: