Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

oozie launcher logs

Highlighted

oozie launcher logs

Contributor

Hello All,

 

By default oozie launcher logs (jobhistory logs)get stored in /tmp/logs location in HDFS. Is there any way to store those logs in local filesystem instead of HDFS?

 

Kindly suggest.

 

Thanks,

Priya

1 REPLY 1

Re: oozie launcher logs

Master Guru
Oozie Launchers run as YARN jobs (single map-task MR job with a dummy input) and therefore their scheduling and logging all follow standard YARN methods, i.e. logs from whichever NodeManager the container got to run on gets uploaded to HDFS as part of YARN's log aggregation step when the container execution is completed.

You can download these logs to local filesystem by running 'yarn logs -applicationId <app ID of launcher job>' for example, but they cannot be written to a single local filesystem as the logging itself is occurring in a distributed environment.

If you need local-like-access to HDFS files for your scripts/etc. to discover these logs, you can consider mounting HDFS via NFS: https://www.cloudera.com/documentation/enterprise/latest/topics/admin_hdfs_nfsgateway.html