Support Questions

Find answers, ask questions, and share your expertise

Generateflow file not genarating flow files

Rising Star

Hi ,

I am trying to generate a sample flow file using generateflowfile processor . i have scheduled it for 5 secs . i don't see any file getting generated . Does this need any additional grants . please find the screenshot of the properties .

sample flow :-

1 ACCEPTED SOLUTION

Master Guru
@Bharadwaj Bhimavarapu

Along the menu bar at the top of the NiFi UI is there is a field that shows the current number of active threads in your NiFi.

16644-screen-shot-2017-06-30-at-21827-pm.png

Is this a standalone NIFi install or a multi-node NIFi cluster?

Under "Controller Settings" found within the hamburger menu in the upper right corner of the UI, What do you have configured for "Maximum Timer Driven Thread Count" and " Maximum Event Driven Thread Count"?

I am wondering if you have some other processors in your NIFi holding all your available threads so this processor cannot get one.

Thanks,

Matt

View solution in original post

10 REPLIES 10

Contributor

@Bharadwaj Bhimavarapu There are no attachments to the question. Can you please check again? Do you see any error message on the processor or in the bulletin board, or in nifi-app.log?

Rising Star

@Shashank Chandhok for some reason i am not able to upload . But its an simple generateflowfile processor . I don't see any errors neither in bulliton board nor in app logs .

Rising Star

Master Guru

@Bharadwaj Bhimavarapu

Can you also share a screenshot of your GenerateFlowFile "scheduling" and "settings" tabs as well. Is this the only flow on your graph? Anything odd in the nifi-app.log when you start the processor?

Rising Star

@Matt Clarke

Please find the screen shots attached. I don'tcapture3.pngcapture4.png see any thing in app.logs about this processor .

@Bharadwaj Bhimavarapu

If you want to have the processor run every 5 seconds, then you would set that using property "Run Schedule" under the Schedule tab like this

16643-screen-shot-2017-06-30-at-125022-pm.png

Rising Star

@Wynner after making the change also .. i dont see any flowfiles getting genarated .

Master Guru
@Bharadwaj Bhimavarapu

Along the menu bar at the top of the NiFi UI is there is a field that shows the current number of active threads in your NiFi.

16644-screen-shot-2017-06-30-at-21827-pm.png

Is this a standalone NIFi install or a multi-node NIFi cluster?

Under "Controller Settings" found within the hamburger menu in the upper right corner of the UI, What do you have configured for "Maximum Timer Driven Thread Count" and " Maximum Event Driven Thread Count"?

I am wondering if you have some other processors in your NIFi holding all your available threads so this processor cannot get one.

Thanks,

Matt

Rising Star

@Matt Clarke

Do we have any caluclation on how to set those numbers .

Master Guru

@Bharadwaj Bhimavarapu

General guidance here is these values should be set to 2 times the number of available cores and no more then 4 times the number of available cores on a single instance of NiFi. If you are running a NiFi cluster, these values are enforced per node. So a setting of 16 in a 4 node cluster equates to a total of 64 threads cross the cluster. Setting values to high just results in many more threads in cpu wait and will not help performance at all.

Beyond increasing these value you need to be mindful of how many concurrent task you assign each of your processors. Some processor are more cpu intensive then others (meaning they take longer to complete a job holding the thread much longer). You can look at the "tasks/time =: stats on a processor to see if it thread are long or short running. For processors that have long running threads you want to be extra careful on how many concurrent tasks you assign them.

Thanks,

Matt

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.