Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to purge Logs from Cloudera VM

How to purge Logs from Cloudera VM

Explorer

Hi,

   How to purge the map reduce logs that is getting created as per my Programs and Application.

   I executed so Oozie Jobs,

   Where are those logs getting created and how do I purge the old logs.

 

Thanks

3 REPLIES 3

Re: How to purge Logs from Cloudera VM

Champion

@dmishraoc

 

Step1: CM -> Yarn -> Configuration -> Search for the below two parameters
JobHistory Server Maximum Log File Backups : <Default value: 10>
JobHistory Server Max Log Size : <Default Value: 200 MiB>

 

Step2: CM -> Yarn -> Instances -> Get the hostname for JobHistory Server


Step3: Login to the above host and go to the path /var/log/hadoop-mapreduce
Note: If you have 10 history file + one current file and each file size is 201M, then you are good. It is automatically purging & you don't need to purge anything

 

Thanks

Kumar

Re: How to purge Logs from Cloudera VM

Explorer

Can you please give me the steps without going to Cloudera Manager.

I am not opening cloudera manager as I have less amount of Ram.

 

Thanks

 

Highlighted

Re: How to purge Logs from Cloudera VM

Champion

@dmishraoc

 

You can get the parameters mentioned in step 1 & 2 from yarn-site.xml

 

and follow the step3: go to the path /var/log/hadoop-mapreduce
Note: If you have 10 history file + one current file and each file size is 201M, then you are good. It is automatically purging & you don't need to purge anything