Support Questions
Find answers, ask questions, and share your expertise

Fetchfile processor in nifi

Highlighted

Fetchfile processor in nifi

For handling parallel sessions in fetchfile processor i increased the run schedule and concurrent tasks in fetchfile processor..there would be around 3k files which needs to be handled parallely..and there would be files of large size(around 1 gb)..So if we increase the run schedule..then it would take more time to execute..is there any way to tackle that?

3 REPLIES 3
Highlighted

Re: Fetchfile processor in nifi

@Gillu Varghese

"Run Schedule" does not determines the execution time of a processor. Rather it is for scheduling purpose. For example, following settings will result in the execution of your processor every 10 sec.

70381-screen-shot-2018-04-16-at-15952-pm.png

And every time your processor run, it shall pick up all the files in the mentioned directory.

The run schedule has nothing to do with the efficiency of your flow. Use it for scheduling your flow accordingly. Setting the number of "Concurrent tasks" will result in multiple threads taking care of data retrieval for you and you should be fine.

Highlighted

Re: Fetchfile processor in nifi

@Rahul Soni

Thanks for the information.it was really helpful.

Highlighted

Re: Fetchfile processor in nifi

@Gillu Varghese Please mark the answer as accepted if it solved your query!

Don't have an account?