Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How can I control the number of threads executing dataflows in a Nifi instance ?

Solved Go to solution

How can I control the number of threads executing dataflows in a Nifi instance ?

New Contributor
 
1 ACCEPTED SOLUTION

Accepted Solutions

Re: How can I control the number of threads executing dataflows in a Nifi instance ?

Super Guru

Hi @Tarek Elgamal,

Click on Global menu and Click on Controller settings button then in General tab from this tab you can control threads running in NiFi instance.

42565-nifi-control.png

42566-nifi-control2.png

In addition you can refer to below community links

https://community.hortonworks.com/questions/76117/how-to-improve-nifi-concurrency.html

4 REPLIES 4

Re: How can I control the number of threads executing dataflows in a Nifi instance ?

Super Guru

Hi @Tarek Elgamal,

Click on Global menu and Click on Controller settings button then in General tab from this tab you can control threads running in NiFi instance.

42565-nifi-control.png

42566-nifi-control2.png

In addition you can refer to below community links

https://community.hortonworks.com/questions/76117/how-to-improve-nifi-concurrency.html

Re: How can I control the number of threads executing dataflows in a Nifi instance ?

New Contributor

Thanks @Shu , thats what I was looking for , do you have an idea what is the corresponding parameter in nifi.properties file?

Re: How can I control the number of threads executing dataflows in a Nifi instance ?

Super Guru

@Tarek Elgamal,
I don't think corresponding parameters are in nifi.properties file and these params are for each node not for the whole cluster.The concurrent tasks assigned to your processors pull threads from the Maximum Timer Driven Thread Count pool.

You can refer to below community links to get more info about these parameters

https://community.hortonworks.com/questions/104718/handlehttprequest-configure-to-process-millions-o...
https://community.hortonworks.com/questions/140889/in-nifi-all-the-processors-in-the-data-flow-are-e...
https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#system_properties

Highlighted

Re: How can I control the number of threads executing dataflows in a Nifi instance ?

New Contributor

Thanks @Shu that was really helpful.. I am just wondering that when I set the number of threads of the whole instance to 1 and I have two processors connected to each other they still manage to somehow run concurrently .. The first processor takes on average 2.5 seconds per input and the second processors takes on average 4.5 seconds.. I gave it 100 inputs and I was expecting it to finish in around 700 seconds (i.e., sequential execution) but it still manages to finish in 480 seconds which suggests that each processor is using a separate thread and they do not wait on each other. Am I missing something here ?

Don't have an account?
Coming from Hortonworks? Activate your account here