About jpconver2

jpconver2 · ‎01-13-2023

Dear community, We have a NiFi cluster installation of 3 nodes, and we need to call on an API that demands to be called, at most, once every 5 seconds. We found the ControlRate processor which seems exactly what we need, but at our 3 nodes installation ends up doing 3 calls every 5 seconds. Is ControlRate only meant to be used on a single node installation, or on flows that start from a processor set to Primary Only? Is there any way to use ControlRate in a multi node installation, or any combination of processors that would allow us to get that result? We are running NiFi 1.7 . 1 but if there is any solution in a newer NiFi we may be able to backport it. Thanks a lot for any advice on this. Best Regards!! JP

jpconver2 · ‎11-16-2020

Thanks a lot for your answer!!

jpconver2 · ‎11-11-2020

Dear community, We have a NiFi cluster installation of 3 nodes with 30GB of RAM and 15 CPUs each. The NiFi version that we are using is the 1.7.1 and we are running 43.000 NiFi processors in the cluster. The question is if there is a known limitation of the amount of processors that we can run in the same cluster considering that we can scale horizontally and also add more CPU and memory. Also if somebody is running more processors in the same cluster or have experience in this topic that would be a great piece of information. Thanks a lot for any advice on this Best Regards! Juan Pablo

Online	Offline
Last Visited	‎01-13-2023 06:31 PM

Member Since	‎08-14-2018 08:15 PM
Last Visited	‎01-13-2023 06:31 PM
Posts	6

Cloudera Community

Cloudera Community

Rate limit in a clusterized NiFi installation

Re: NiFi cluster processor count scalability

NiFi cluster processor count scalability