@Allan Batista-
Each NiFi node in your 10 node cluster is running with its own copy of the flow.xml and operating against its own set of FlowFiles. Concurrent tasks setting on a component (for example processor) is applied to every node. In order for a NiFi node to execute a component it must have the ability to request a thread from the controller, so it must have at least 1 concurrent task. So setting 1 concurrent task (default) on a processor means that every node can run that processor with one concurrent task (so 10 potential threads in operation across cluster). Threads are not shared between nodes.
-
Also keep in mind that setting 6 concurrent tasks on a single component (60 equivalent across c10 node cluster) does not mean that that component on each node will always use all 6. If the volume of data being processed by the component does not warrant using more then 1 or 2 concurrent tasks, the other concurrent tasks will not be utilized.
-
This article may help here:
https://community.hortonworks.com/articles/221808/understanding-nifi-max-thread-pools-and-processor....
-
Thank you,
Matt
-
If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link.
