Support Questions

Find answers, ask questions, and share your expertise

How to limit the number of records fetched by Solr in response to a query to improve search performance?

avatar
Explorer

I have 1 Lac records indexing per day and I want to query records for last 7 days only. As per my understanding, when we query solr it fetches all the records with respect to the query and sort them in some order, in memory, and provides us the required number of rows.

As I only want last 7 lac records, can they be fetched without touching the old records? What is the best way to do it? 

3 REPLIES 3

avatar
Master Collaborator

A Solr query can have a filter clause that will ensure only the documents from the last 7 days are fetched. What you are looking for is a filter query (fq) parameter.  For example you could add this to your query:

 

&fq=createdate:[NOW-7DAY/DAY TO NOW]

 

 

You can read more about filtering in documentation here.  

 

If this is helpful please don't forget to give kudos and/or accept solution.

avatar
Explorer

hi @aakulov 

Thanks for your reply.

I think I was not able to clearly explain the problem.

Let's take an example, suppose I have 1M records for phone numbers. Here the frequency of phone number P1 is x and P2 is 2x (2 times x).
My observation is that time taken in fetching same number of rows, let's say 10k, for both phone numbers is different  (QTime is more in case of P2) even if I don't use any sort parameter. I want the QTime to be same if number of rows in both the cases is same.

avatar
Master Collaborator

Thanks for clarifying the question, but I'm afraid I still don't know what you are trying to achieve. Based on your example I understand you have 10K records/documents with phone number P1 and 20K records/documents with phone number P2. You are retrieving all 10K documents only in a single query? And you want the performance of 10K row P1 query to be the same as a 10K row P2 query. Is that right?

 

Solr was never built to retrieve large number of objects at one time. It's meant for faceted-search that returns humanly consumable number of records in the result set (see pagination). Are you doing this for UI display or for integration purposes? There is some useful documentation here on getting large number of records from Solr. 

 

It would be helpful if you shared your query, what your data structure is, and what is the use case. That way the community can better understand the problem and provide potential solution.