Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Generateflow file not genarating flow files

avatar
Expert Contributor

Hi ,

I am trying to generate a sample flow file using generateflowfile processor . i have scheduled it for 5 secs . i don't see any file getting generated . Does this need any additional grants . please find the screenshot of the properties .

sample flow :-

1 ACCEPTED SOLUTION

avatar
Super Mentor
@Bharadwaj Bhimavarapu

Along the menu bar at the top of the NiFi UI is there is a field that shows the current number of active threads in your NiFi.

16644-screen-shot-2017-06-30-at-21827-pm.png

Is this a standalone NIFi install or a multi-node NIFi cluster?

Under "Controller Settings" found within the hamburger menu in the upper right corner of the UI, What do you have configured for "Maximum Timer Driven Thread Count" and " Maximum Event Driven Thread Count"?

I am wondering if you have some other processors in your NIFi holding all your available threads so this processor cannot get one.

Thanks,

Matt

View solution in original post

10 REPLIES 10

avatar
Super Mentor

@Bharadwaj Bhimavarapu

General guidance here is these values should be set to 2 times the number of available cores and no more then 4 times the number of available cores on a single instance of NiFi. If you are running a NiFi cluster, these values are enforced per node. So a setting of 16 in a 4 node cluster equates to a total of 64 threads cross the cluster. Setting values to high just results in many more threads in cpu wait and will not help performance at all.

Beyond increasing these value you need to be mindful of how many concurrent task you assign each of your processors. Some processor are more cpu intensive then others (meaning they take longer to complete a job holding the thread much longer). You can look at the "tasks/time =: stats on a processor to see if it thread are long or short running. For processors that have long running threads you want to be extra careful on how many concurrent tasks you assign them.

Thanks,

Matt