Ranger Audit Retention

[#1]

Hi All,

I've searched around HCC and was unable to come up with an answer to: There doesn't seem to be a way to automatically define a retention policy for the Ranger Audit Data (audit data is kept indefinitely unless we manually remove it). Is there a plan to add an automatic retention policy for these audit logs in HDFS and/or Solr?

* Falcon can be used for retention in HDFS - but will there be an easy-to-configure option under Ambari>Ranger under Audit?

[#2]

Ranger stores access audit logs in HDFS for long term archival. Retention period depends on compliance requirements of the customer; there is no automatic purging of audit log files in HDFS.

[#3 · Accepted solution]

Hi @Ryan Cicak

The best practice is to configure Ranger audits to both Solr and HDFS. HDFS is used for long term audit storage so you won't want to delete audit data. Solr should be used for short term storage. By using Solr you have data indexed and you can query it quickly from Ranger UI. I am not aware of any setting or property in Ranger to set a TTL and automatically delete data.

You may leverage Solr TTL feature to purge data (link) or schedule a job to issue a delete query periodically.