Reply
Highlighted
New Contributor
Posts: 5
Registered: ‎08-11-2015

Oozie not cleaning up old jobs from Oozie database

[ Edited ]

We are using CDH 4.5.0 alongwith Cloudera Manager 5.2.0 and Oozie 3.3.2.


The issue we previously faced is mentioned here -> 

https://community.cloudera.com/t5/Batch-Processing-and-Workflow/Oozie-not-cleaning-up-old-jobs-from-...

 

To circumvent the issue we killed our long running co-ordinators.

 

The config is as such,

oozie.services.ext - org.apache.oozie.service.PurgeService
oozie.service.PurgeService.older.than - 15
oozie.service.PurgeService.coord.older.than - 7
oozie.service.PurgeService.bundle.older.than - 7
oozie.service.PurgeService.purge.interval - 60

 

 

I have kept the logs in DEBUG mode to see the Purge Command in the logs.

 

There are 2 issues now,

 

  1. The co-ordinators were KILLED on Aug 18 23:45:45 PDT 2015 BUT Oozie has not yet purged them.
    Nor do I observe any decrease in the no of workflow jobs /actions in the database. I don't see any 
    DELETE command for oozie user in 'show processlist' for MySQL.
  2. I can't see any Purge command running in the logs (in DEBUG mode) which I used to see previously.There have been no 
    config changes which can cause the Purge service to stop.

How can I tackle this issue?

Posts: 1,695
Kudos: 341
Solutions: 264
Registered: ‎07-31-2013

Re: Oozie not cleaning up old jobs from Oozie database

Could you share the coordinator info output? Are there any pending instances still in running state?

If you switch to debug logging, you should see some messages when there's a deletion service run with strings such as: "STARTED Coord-Purge to purge Jobs older than". Do you observe such a thing?
New Contributor
Posts: 5
Registered: ‎08-11-2015

Re: Oozie not cleaning up old jobs from Oozie database

Hi Harsh,

 

The problem is I don't see the messages related to Purging in Oozie Logs (in DEBUG mode).

The KILLED Co-ordinators don't have any pending workflows.

Infact, even today the entries related to the KILLED co-ordinator are not purged from the database.

Announcements