Hi @Matt Clarke, Sorry I'm new to Nifi and didn't realise the controller settings for the Process Group. It seems as though increasing the "Maximum Timer Driven Thread Count" increased the transfer rate between Process Groups. This is very confusing, I thought the FlowFile transfer was event driven. Update: It seems as though in the Source Code that the FlowFile transfer is TimeDriven. Thanks a lot for the information by the way. ... View more

Hi @Matt Clarke, I don't believe it is the swapping that is an issue. My backpressure threshold values are all set to 10,000. Even when my queue is below the swapping threshold the rate at which the FlowFiles are being transferred from one Process Group to another are abysmal. I've been doing some digging and found that FlowFiles from a Process Group are transferred in batches of 100. This seems to be in the source code LocalPort.java. I'm not quite sure what would be limiting the frequency of this though. I think if I disband the Process Group and remove the Nifi internal port to port layer this may remove the latency. But this is just a hassle and makes Process Groups redundant. ... View more