I would like to know how to control the number of flow files received by a nifi client using site to site RAW protocol ?
I am planning to setup 2 nifi instances the second one will be getting the data from first (2nd one pulls the data from 1st), The second nifi instance will be running in a machine with lower memory. what configurations should i make so that the second one will pull the data in a controlled manner & wont result in an outofmemory error.
I am trying to write an external tool to pull data from nifi using site to site raw protocol.
How to controll the number of flow files recieved by my client in a single transaction ?
Thanks for the response. But i am more interested to know the configurations to be done at the reciever end than at the producer end.
What you could do is to play with the combination of backpressure parameters and scheduling frequency for the first processor after the reception of the data. Or just use the ControlRate processor. When backpressure is enabled between the RPG and ControlRate, the RPG will stop pulling the data from the remote system.