Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to purge Logs from Cloudera VM

avatar
Explorer

Hi,

   How to purge the map reduce logs that is getting created as per my Programs and Application.

   I executed so Oozie Jobs,

   Where are those logs getting created and how do I purge the old logs.

 

Thanks

3 REPLIES 3

avatar
Champion

@dmishraoc

 

Step1: CM -> Yarn -> Configuration -> Search for the below two parameters
JobHistory Server Maximum Log File Backups : <Default value: 10>
JobHistory Server Max Log Size : <Default Value: 200 MiB>

 

Step2: CM -> Yarn -> Instances -> Get the hostname for JobHistory Server


Step3: Login to the above host and go to the path /var/log/hadoop-mapreduce
Note: If you have 10 history file + one current file and each file size is 201M, then you are good. It is automatically purging & you don't need to purge anything

 

Thanks

Kumar

avatar
Explorer

Can you please give me the steps without going to Cloudera Manager.

I am not opening cloudera manager as I have less amount of Ram.

 

Thanks

 

avatar
Champion

@dmishraoc

 

You can get the parameters mentioned in step 1 & 2 from yarn-site.xml

 

and follow the step3: go to the path /var/log/hadoop-mapreduce
Note: If you have 10 history file + one current file and each file size is 201M, then you are good. It is automatically purging & you don't need to purge anything