Support Questions
Find answers, ask questions, and share your expertise

solr process keeps stopping

New Contributor

Solr process keeps stopping each 3 days

I am getting the below error

22/02/27 05:15:21 ERROR handler.RequestHandlerBase: [qtp417301556-3371]: org.apache.solr.common.SolrException: Exception writing document id 6ae68838-4669-4f11-9544-53d22025085c-5 to the index; possible analysis error: number of documents in the index cannot exceed 2147483519
Caused by: java.lang.IllegalArgumentException: number of documents in the index cannot exceed 2147483519

 

I have 1 solar server, Cloudera version 7.4.4, solar version 8.4.1

what do you suggest to solve the problem?

Thanks

 

1 ACCEPTED SOLUTION

Super Collaborator

Hello @Almolki 

 

Thanks for using Cloudera Community. Based on the Post, Your Team is experiencing [1] on Solr 8.4.1. 

 

The Issue being faced by your Team is a Known Limitation of Solr, wherein each Shard can host 2B Documents Maximum [2]. Your Team have following Choices:

  • Use SplitShard to split the Shard with 2B Documents into 2 Daughter Shards. Refer [3] for using SplitShard API. 
  • If your Team have the Source Data, Create a Collection afresh with Higher Shards & Reindex the Source Data afresh. 

Kindly review & let us know if your Queries have been addressed. 

 

Regards, Smarak

 

[1] Caused by: java.lang.IllegalArgumentException: number of documents in the index cannot exceed 2147483519

[2] https://issues.apache.org/jira/browse/SOLR-3504

[3] https://solr.apache.org/guide/8_4/shard-management.html#shard-management

 

View solution in original post

3 REPLIES 3

New Contributor

any help?

Super Collaborator

Hello @Almolki 

 

Thanks for using Cloudera Community. Based on the Post, Your Team is experiencing [1] on Solr 8.4.1. 

 

The Issue being faced by your Team is a Known Limitation of Solr, wherein each Shard can host 2B Documents Maximum [2]. Your Team have following Choices:

  • Use SplitShard to split the Shard with 2B Documents into 2 Daughter Shards. Refer [3] for using SplitShard API. 
  • If your Team have the Source Data, Create a Collection afresh with Higher Shards & Reindex the Source Data afresh. 

Kindly review & let us know if your Queries have been addressed. 

 

Regards, Smarak

 

[1] Caused by: java.lang.IllegalArgumentException: number of documents in the index cannot exceed 2147483519

[2] https://issues.apache.org/jira/browse/SOLR-3504

[3] https://solr.apache.org/guide/8_4/shard-management.html#shard-management

 

New Contributor

Hi @smdas 

Thanks for your help


this limit can not be increased, by configuration?

 

best regards

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.