Reply
Explorer
Posts: 29
Registered: ‎01-20-2017

How to purge Logs from Cloudera VM

Hi,

   How to purge the map reduce logs that is getting created as per my Programs and Application.

   I executed so Oozie Jobs,

   Where are those logs getting created and how do I purge the old logs.

 

Thanks

Posts: 390
Topics: 11
Kudos: 60
Solutions: 35
Registered: ‎09-02-2016

Re: How to purge Logs from Cloudera VM

@dmishraoc

 

Step1: CM -> Yarn -> Configuration -> Search for the below two parameters
JobHistory Server Maximum Log File Backups : <Default value: 10>
JobHistory Server Max Log Size : <Default Value: 200 MiB>

 

Step2: CM -> Yarn -> Instances -> Get the hostname for JobHistory Server


Step3: Login to the above host and go to the path /var/log/hadoop-mapreduce
Note: If you have 10 history file + one current file and each file size is 201M, then you are good. It is automatically purging & you don't need to purge anything

 

Thanks

Kumar

Explorer
Posts: 29
Registered: ‎01-20-2017

Re: How to purge Logs from Cloudera VM

Can you please give me the steps without going to Cloudera Manager.

I am not opening cloudera manager as I have less amount of Ram.

 

Thanks

 

Highlighted
Posts: 390
Topics: 11
Kudos: 60
Solutions: 35
Registered: ‎09-02-2016

Re: How to purge Logs from Cloudera VM

@dmishraoc

 

You can get the parameters mentioned in step 1 & 2 from yarn-site.xml

 

and follow the step3: go to the path /var/log/hadoop-mapreduce
Note: If you have 10 history file + one current file and each file size is 201M, then you are good. It is automatically purging & you don't need to purge anything

Announcements