Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

solr process keeps stopping

New Contributor

Solr process keeps stopping each 3 days

I am getting the below error

22/02/27 05:15:21 ERROR handler.RequestHandlerBase: [qtp417301556-3371]: org.apache.solr.common.SolrException: Exception writing document id 6ae68838-4669-4f11-9544-53d22025085c-5 to the index; possible analysis error: number of documents in the index cannot exceed 2147483519
Caused by: java.lang.IllegalArgumentException: number of documents in the index cannot exceed 2147483519

 

I have 1 solar server, Cloudera version 7.4.4, solar version 8.4.1

what do you suggest to solve the problem?

Thanks

 

1 ACCEPTED SOLUTION

Super Collaborator

Hello @Almolki 

 

Thanks for using Cloudera Community. Based on the Post, Your Team is experiencing [1] on Solr 8.4.1. 

 

The Issue being faced by your Team is a Known Limitation of Solr, wherein each Shard can host 2B Documents Maximum [2]. Your Team have following Choices:

  • Use SplitShard to split the Shard with 2B Documents into 2 Daughter Shards. Refer [3] for using SplitShard API. 
  • If your Team have the Source Data, Create a Collection afresh with Higher Shards & Reindex the Source Data afresh. 

Kindly review & let us know if your Queries have been addressed. 

 

Regards, Smarak

 

[1] Caused by: java.lang.IllegalArgumentException: number of documents in the index cannot exceed 2147483519

[2] https://issues.apache.org/jira/browse/SOLR-3504

[3] https://solr.apache.org/guide/8_4/shard-management.html#shard-management

 

View solution in original post

3 REPLIES 3

New Contributor

any help?

Super Collaborator

Hello @Almolki 

 

Thanks for using Cloudera Community. Based on the Post, Your Team is experiencing [1] on Solr 8.4.1. 

 

The Issue being faced by your Team is a Known Limitation of Solr, wherein each Shard can host 2B Documents Maximum [2]. Your Team have following Choices:

  • Use SplitShard to split the Shard with 2B Documents into 2 Daughter Shards. Refer [3] for using SplitShard API. 
  • If your Team have the Source Data, Create a Collection afresh with Higher Shards & Reindex the Source Data afresh. 

Kindly review & let us know if your Queries have been addressed. 

 

Regards, Smarak

 

[1] Caused by: java.lang.IllegalArgumentException: number of documents in the index cannot exceed 2147483519

[2] https://issues.apache.org/jira/browse/SOLR-3504

[3] https://solr.apache.org/guide/8_4/shard-management.html#shard-management

 

New Contributor

Hi @smdas 

Thanks for your help


this limit can not be increased, by configuration?

 

best regards