How can I control the number of threads executing dataflows in a Nifi instance ?

[#1 · Accepted solution]

Hi @Tarek Elgamal,

Click on Global menu and Click on Controller settings button then in General tab from this tab you can control threads running in NiFi instance.

In addition you can refer to below community links

https://community.hortonworks.com/questions/76117/how-to-improve-nifi-concurrency.html

[#2]

Thanks @Shu , thats what I was looking for , do you have an idea what is the corresponding parameter in nifi.properties file?

[#3]

@Tarek Elgamal,
I don't think corresponding parameters are in nifi.properties file and these params are for each node not for the whole cluster.The concurrent tasks assigned to your processors pull threads from the Maximum Timer Driven Thread Count pool.

You can refer to below community links to get more info about these parameters

https://community.hortonworks.com/questions/104718/handlehttprequest-configure-to-process-millions-o...
https://community.hortonworks.com/questions/140889/in-nifi-all-the-processors-in-the-data-flow-are-e...
https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#system_properties

[#4]

Thanks @Shu that was really helpful.. I am just wondering that when I set the number of threads of the whole instance to 1 and I have two processors connected to each other they still manage to somehow run concurrently .. The first processor takes on average 2.5 seconds per input and the second processors takes on average 4.5 seconds.. I gave it 100 inputs and I was expecting it to finish in around 700 seconds (i.e., sequential execution) but it still manages to finish in 480 seconds which suggests that each processor is using a separate thread and they do not wait on each other. Am I missing something here ?