Support Questions
Find answers, ask questions, and share your expertise

ranger is not updating access audit logs

New Contributor

Ranger is not updating the access audits in ranger after a certa date solr logs are giving the below error:

ERROR (qtp246273275-120) [c:ranger_audits s:shard1 r:core_node2 x:ranger_audits_shard1_replica_n1] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Exception writing document id 8cb43886-8056-46b2-8b24-0ead48c8280b-1 to the index; possible analysis error: number of documents in the index cannot exceed 2147483519

 

how can this be troubleshooted?

1 ACCEPTED SOLUTION

Cloudera Employee

Hello @clouderaskme ,

 

From the above error message , we could tell that you would be hitting  SOLR-3504. The issue is due to limitation from Solr side where 1 shard can only index upto 2.14 Billion.

The solution would be to create a new ranger_audits collection with 2 shards instead of 1. As it can index more documents.

 

You may also try to delete the older records if the solr instance is still up and running and see if the issue been resolved.

Please modify the http with https if SSL is enabled and check the port as per your environment and run the below command.

curl -ikv --negotiate -u: "http://$(hostname -f):8886/solr/ranger_audits/update?commit=true" -H "Content-Type: text/xml" --data-binary "<delete><query>evtTime:[* TO NOW-15DAYS]</query></delete>"    

 

There is another method of splitting the shard. Please refer to the below doc
https://my.cloudera.com/knowledge/ERROR-quotToo-many-documents-composite-IndexReaders-cannot?id=7473...

View solution in original post

2 REPLIES 2

Cloudera Employee

Hello @clouderaskme ,

 

From the above error message , we could tell that you would be hitting  SOLR-3504. The issue is due to limitation from Solr side where 1 shard can only index upto 2.14 Billion.

The solution would be to create a new ranger_audits collection with 2 shards instead of 1. As it can index more documents.

 

You may also try to delete the older records if the solr instance is still up and running and see if the issue been resolved.

Please modify the http with https if SSL is enabled and check the port as per your environment and run the below command.

curl -ikv --negotiate -u: "http://$(hostname -f):8886/solr/ranger_audits/update?commit=true" -H "Content-Type: text/xml" --data-binary "<delete><query>evtTime:[* TO NOW-15DAYS]</query></delete>"    

 

There is another method of splitting the shard. Please refer to the below doc
https://my.cloudera.com/knowledge/ERROR-quotToo-many-documents-composite-IndexReaders-cannot?id=7473...

Moderator

@clouderaskme Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks!


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:
; ;