@Aprendizado
The GetMongo processor (Assume this is what you are using) utilize a Mongo client library and not something custom written in NiFi. So limiting returns needs to be something that client library supports.
The good news is that Mongo "limit" which is exposed by the processor should work for your use case (never tried this myself).
Example us setting a Mongo limit based on time:
https://stackoverflow.com/questions/8835757/return-query-based-on-date
Now the GetMongo processor does support and inbound connection which means a source FlowFile could be used to trigger each execution of the GetMongo processor. The "limit" property in the GetMongo processor also supports NiFi Expression Language (NEL), which means that the limit could be set dynamically on the source trigger FlowFile and passed to the GetMongo on each execution.
This means that after a successful run, you would need to extract from your Mongo results the date from which next execution needs to start. You could write that date for example to a Distributed Map cache using putDistributedMapCache processor. Then at beginning of your dataflow use a GenerateFlowFile --> FetchDistributedMapCache --> updateAttribute --> GetMongo to retrieve latest date that needs to be put in the limits configuration you pass to the GetMongo processor for next execution. The GenerateFlowFile scheduling controls the execution of this flow, so configure cron to control how often it creates the FlowFile that triggers yorur dataflow.
Hopefully this gives you an idea of how you can accomplish your use case.
If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.
Thank you,
Matt
