- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How to delete log folder faster having files lots of file and heavy log folder
Created 10-18-2016 10:43 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Problem : Deletion of log folder with below details
Log size is around 10gb
No . of files around 9k files.
Simple rm -rf log folder doesn't works well.
Created 10-18-2016 03:36 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If you are not using log4j: If you are looking to delete the files for good then there is not many options available other than rm -rf; however there are few tweaks that you can do to make it faster
- you can perhaps run multiple rm scripts in parallel (multiple threads)
- In order to do this, you should be able to logically separate the log files either by folder or name format
- Once you have done that, you can run multiple rm commands in background like something below
nohup rm -fr app1-2016* > /tmp/nohup.out 2>&1 & nohup rm -fr app1-2015* > /tmp/nohup.out 2>&1 &
If using log4j: You should probably be 'DailyRollingFileAppender" with 'maxBackupIndex' - this will essentially limit the max file size of your log and then purge the older contents. More details here: http://www.codeproject.com/Articles/81462/DailyRollingFileAppender-with-maxBackupIndex
Outside of this, you should consider the below 2 things for future use cases
- Organize the logs by folder (normally broken down like /logs/appname/yyyy/mm/dd/hh/<log files>
- Have a mechanism that will either delete the old log files, or archive it to a different log archive server
Hopefully this helps. If it does, please 'accept' and 'upvote' the answer. Thank you!!
Created 10-18-2016 03:36 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If you are not using log4j: If you are looking to delete the files for good then there is not many options available other than rm -rf; however there are few tweaks that you can do to make it faster
- you can perhaps run multiple rm scripts in parallel (multiple threads)
- In order to do this, you should be able to logically separate the log files either by folder or name format
- Once you have done that, you can run multiple rm commands in background like something below
nohup rm -fr app1-2016* > /tmp/nohup.out 2>&1 & nohup rm -fr app1-2015* > /tmp/nohup.out 2>&1 &
If using log4j: You should probably be 'DailyRollingFileAppender" with 'maxBackupIndex' - this will essentially limit the max file size of your log and then purge the older contents. More details here: http://www.codeproject.com/Articles/81462/DailyRollingFileAppender-with-maxBackupIndex
Outside of this, you should consider the below 2 things for future use cases
- Organize the logs by folder (normally broken down like /logs/appname/yyyy/mm/dd/hh/<log files>
- Have a mechanism that will either delete the old log files, or archive it to a different log archive server
Hopefully this helps. If it does, please 'accept' and 'upvote' the answer. Thank you!!