Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

solr process keeps stopping

avatar
New Contributor

Solr process keeps stopping each 3 days

I am getting the below error

22/02/27 05:15:21 ERROR handler.RequestHandlerBase: [qtp417301556-3371]: org.apache.solr.common.SolrException: Exception writing document id 6ae68838-4669-4f11-9544-53d22025085c-5 to the index; possible analysis error: number of documents in the index cannot exceed 2147483519
Caused by: java.lang.IllegalArgumentException: number of documents in the index cannot exceed 2147483519

 

I have 1 solar server, Cloudera version 7.4.4, solar version 8.4.1

what do you suggest to solve the problem?

Thanks

 

1 ACCEPTED SOLUTION

avatar
Super Collaborator

Hello @Almolki 

 

Thanks for using Cloudera Community. Based on the Post, Your Team is experiencing [1] on Solr 8.4.1. 

 

The Issue being faced by your Team is a Known Limitation of Solr, wherein each Shard can host 2B Documents Maximum [2]. Your Team have following Choices:

  • Use SplitShard to split the Shard with 2B Documents into 2 Daughter Shards. Refer [3] for using SplitShard API. 
  • If your Team have the Source Data, Create a Collection afresh with Higher Shards & Reindex the Source Data afresh. 

Kindly review & let us know if your Queries have been addressed. 

 

Regards, Smarak

 

[1] Caused by: java.lang.IllegalArgumentException: number of documents in the index cannot exceed 2147483519

[2] https://issues.apache.org/jira/browse/SOLR-3504

[3] https://solr.apache.org/guide/8_4/shard-management.html#shard-management

 

View solution in original post

3 REPLIES 3

avatar
New Contributor

any help?

avatar
Super Collaborator

Hello @Almolki 

 

Thanks for using Cloudera Community. Based on the Post, Your Team is experiencing [1] on Solr 8.4.1. 

 

The Issue being faced by your Team is a Known Limitation of Solr, wherein each Shard can host 2B Documents Maximum [2]. Your Team have following Choices:

  • Use SplitShard to split the Shard with 2B Documents into 2 Daughter Shards. Refer [3] for using SplitShard API. 
  • If your Team have the Source Data, Create a Collection afresh with Higher Shards & Reindex the Source Data afresh. 

Kindly review & let us know if your Queries have been addressed. 

 

Regards, Smarak

 

[1] Caused by: java.lang.IllegalArgumentException: number of documents in the index cannot exceed 2147483519

[2] https://issues.apache.org/jira/browse/SOLR-3504

[3] https://solr.apache.org/guide/8_4/shard-management.html#shard-management

 

avatar
New Contributor

Hi @smdas 

Thanks for your help


this limit can not be increased, by configuration?

 

best regards