question HDFS Audit Log File Size Issues in Support Questions

HDFS Audit Log File Size Issues

ramcharantej — Thu, 16 Mar 2017 17:10:50 GMT

Hi All,

HDFS Audit logs size have become huge since recent past days, on weekends /root was completely occupied, upon checking largest files in /root it was hdfs.audit.log file which were sizing between 300 MB to 9.9 GB. When I checked file sizes from Oldest to Newest, oldest were 300 MB- 500MB sized, the Newest which are from last 6 days were sizing between 1.9 GB to 9.9 GB! Are there any particular reason for huge space occupancy all of a sudden?

Ps: We have not installed Ranger, KNOX, SOLR and ATLAS.

Re: HDFS Audit Log File Size Issues

namaheshwari — Mon, 08 Jul 2024 12:30:13 GMT

You should use the following properties of the RollingFileAppender to efficiently control the Size and the number of backup index of old log files:

maxFileSize:This is the critical size of the file above which the file will be rolled. Default value is 10 MB.

maxBackupIndex: This property denotes the number of backup files to be created. Default value is 1.

More details in :

https://community.cloudera.com/t5/Community-Articles/How-to-control-size-of-log-files-for-various-HDP-components/ta-p/245063

Re: HDFS Audit Log File Size Issues

namaheshwari — Mon, 08 Jul 2024 12:31:48 GMT

This posts also explains similar issue:

https://community.cloudera.com/t5/Support-Questions/How-to-rotate-and-archive-hdfs-audit-log-file/m-p/137118

Re: HDFS Audit Log File Size Issues

ramcharantej — Mon, 20 Mar 2017 17:24:10 GMT

Thanks Namit, this worked for me on my Dev Environment. Will try on next Change on Prod too. Thanks.