Skip to main content
clouderaskme
Silver ⭐️
Jun 1, 2022 9:00 PM UTC
Solved

ranger is not updating access audit logs

Ranger is not updating the access audits in ranger after a certa date solr logs are giving the below error:

ERROR (qtp246273275-120) [c:ranger_audits s:shard1 r:core_node2 x:ranger_audits_shard1_replica_n1] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Exception writing document id 8cb43886-8056-46b2-8b24-0ead48c8280b-1 to the index; possible analysis error: number of documents in the index cannot exceed 2147483519

 

how can this be troubleshooted?

Tylenol
TylenolAnswer
Gold ⭐️
Jun 2, 2022 2:21 AM UTC

Hello @clouderaskme ,

 

From the above error message , we could tell that you would be hitting  SOLR-3504. The issue is due to limitation from Solr side where 1 shard can only index upto 2.14 Billion.

The solution would be to create a new ranger_audits collection with 2 shards instead of 1. As it can index more documents.

 

You may also try to delete the older records if the solr instance is still up and running and see if the issue been resolved.

Please modify the http with https if SSL is enabled and check the port as per your environment and run the below command.

curl -ikv --negotiate -u: "http://$(hostname -f):8886/solr/ranger_audits/update?commit=true" -H "Content-Type: text/xml" --data-binary "<delete><query>evtTime:[* TO NOW-15DAYS]</query></delete>"

 

There is another method of splitting the shard. Please refer to the below doc
https://my.cloudera.com/knowledge/ERROR-quotToo-many-documents-composite-IndexReaders-cannot?id=74738

2 replies

Tylenol
Gold ⭐️
TylenolAnswer
Gold ⭐️
Jun 2, 2022 2:21 AM UTC

Hello @clouderaskme ,

 

From the above error message , we could tell that you would be hitting  SOLR-3504. The issue is due to limitation from Solr side where 1 shard can only index upto 2.14 Billion.

The solution would be to create a new ranger_audits collection with 2 shards instead of 1. As it can index more documents.

 

You may also try to delete the older records if the solr instance is still up and running and see if the issue been resolved.

Please modify the http with https if SSL is enabled and check the port as per your environment and run the below command.

curl -ikv --negotiate -u: "http://$(hostname -f):8886/solr/ranger_audits/update?commit=true" -H "Content-Type: text/xml" --data-binary "<delete><query>evtTime:[* TO NOW-15DAYS]</query></delete>"

 

There is another method of splitting the shard. Please refer to the below doc
https://my.cloudera.com/knowledge/ERROR-quotToo-many-documents-composite-IndexReaders-cannot?id=74738

Translate
DianaTorres
Community Manager
Community Manager
Jun 6, 2022 5:56 PM UTC

@clouderaskme Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks!

Translate
Regards, Diana Torres, Senior Community Moderator