There isn't enough information in your question to provide you with an accurate answer.   However, note that the FetchFile processor is intended to work with an upstream ListFile processor, which provides you with the opportunity to specify a location and other filtering criteria such as file name patterns or particular sub folders (path pattern) amongst other options.  By default, the ListFile processor outputs one empty FlowFile with attributes related to each file it identifies, these  are then used as an input for the FetchFile so it knows which files to retrieve.  Also, the ListFile is aware of the of its last listing, and will only list files younger than this on it's next execution.   Hope that helps.   The List & Fetch pattern is very common, however should you have simpler requirements, the GetFile processor may be suitable. ... View more

Thanks a lot ... View more

@sangee    Details around your use case may be helpful. The SplitText processor will output all FlowFiles produced at the exact same time to the "splits" relationship.  So if you intent is to wait until all splits from a single source FlowFile are produced before processing those splits, this flow is not needed. Pierre has written an excellent blog around using wait and notify processors in a dataflow that does merge and split.  Check it out here: https://pierrevillard.com/2018/06/27/nifi-workflow-monitoring-wait-notify-pattern-with-split-and-merge/   If you found this helpful with your query, please take a moment to login and click accept on this solution. Thanks, Matt ... View more

Add more RAM and CPU to your NiFi server Add more NiFi servers to your cluster.   How much is HUGE DATA?    You will need some amount of RAM to process it.   Make sure minimum JVM RAM on each node is a good size for big workloads.   32G at least. ... View more