Skip to main content
caisch
caisch
Silver ⭐️⭐️
June 21, 2022
Solved

Is Solr audit retention time configurable by service in CDP 7 or future releases?

Is there any way to e.g. keep the ttl for the service HDFS at 90 DAYS, and set the ttl for the other services to 14 DAYS?

 

---

Related articles:

Support Question "How to define the retention period for Ranger Audits in CDP INFRA Solr?"

https://community.cloudera.com/t5/Support-Questions/How-to-define-the-retention-period-for-Ranger-Audits-in-CDP/td-p/318759

 

solrctl Reference

https://docs.cloudera.com/runtime/7.1.0/search-managing/topics/search-solrctl-ref.html

smdas
smdasAnswer
Gold ⭐️⭐️
June 22, 2022

Hello @caisch 

 

Thanks for using Cloudera Community. Based on your Post, you wish to confirm if TTL for HDFS can be set to 90Days & other Services' TTL is set to 14Days.

 

Since you selected Solr, Let me answer your Post & you can let me know if I understood the Post differently. In Solr, We have Collections which may have Data on HDFS or Local. For Example, RangerAudits Collection may have Data on HDFS & Atlas Collection may have Data on Local. At each Collection Level, the "solrconfig.xml" captures the TTL [1] via DocExpirationUpdateProcessorFactory Class. You can configure the TTL at each Collection Level in Solr & they would Cleanup the Underlying  Data, be it on HDFS or Local. Using the above Example of RangerAudits using HDFS Storage & Atlas using Local Storage, We can set RangerAudits to expire at 90 Days & Atlas to expire at 14 Days, which in turn would remove the Underlying  Data from HDFS & Local respectively for RangerAudits & Atlas. 

 

Kindly review & let us know if the above answers your Post. If No, You may clarify & we shall get back to you accordingly.

 

Regards, Smarak

 

[1] https://solr.apache.org/docs/8_4_0/solr-core/org/apache/solr/update/processor/DocExpirationUpdateProcessorFactory.html

 

View in thread

3 replies

smdas
Gold ⭐️⭐️
smdasAnswer
Gold ⭐️⭐️
June 22, 2022

Hello @caisch 

 

Thanks for using Cloudera Community. Based on your Post, you wish to confirm if TTL for HDFS can be set to 90Days & other Services' TTL is set to 14Days.

 

Since you selected Solr, Let me answer your Post & you can let me know if I understood the Post differently. In Solr, We have Collections which may have Data on HDFS or Local. For Example, RangerAudits Collection may have Data on HDFS & Atlas Collection may have Data on Local. At each Collection Level, the "solrconfig.xml" captures the TTL [1] via DocExpirationUpdateProcessorFactory Class. You can configure the TTL at each Collection Level in Solr & they would Cleanup the Underlying  Data, be it on HDFS or Local. Using the above Example of RangerAudits using HDFS Storage & Atlas using Local Storage, We can set RangerAudits to expire at 90 Days & Atlas to expire at 14 Days, which in turn would remove the Underlying  Data from HDFS & Local respectively for RangerAudits & Atlas. 

 

Kindly review & let us know if the above answers your Post. If No, You may clarify & we shall get back to you accordingly.

 

Regards, Smarak

 

[1] https://solr.apache.org/docs/8_4_0/solr-core/org/apache/solr/update/processor/DocExpirationUpdateProcessorFactory.html

 

Translate
VidyaSargur
Community Manager
VidyaSargur
Community Manager
July 4, 2022

@caisch, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. 

Translate
Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
smdas
Gold ⭐️⭐️
smdas
Gold ⭐️⭐️
July 6, 2022

Hello @caisch 

 

We hope our Post on 06/21 help answer your query concerning Solr TTL. As such, We shall mark the Post as Resolved. If you have any concerns, Feel free to update the Post & we shall get back to you accordingly.

 

Regards, Smarak

Translate
Terms & ConditionsAccessibility statement