Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Capacity planning for NiFi cluster

avatar

Is there any guide available for NiFi capacity planning ? What load characteristic should be looked at ?

- Number of concurrent processors running

- Total throughput

Any help is greatly appreciated.

1 ACCEPTED SOLUTION

avatar
Master Mentor
11 REPLIES 11

avatar

Thank You @mclark This is exactly what I was looking for.

avatar
Super Mentor

The NCM in a NiFi cluster typically needs more heap memory. The number of components (processors, input ports, output ports and relationships) x the number of nodes in the NiFi cluster on the graph will drive how much memory your NCM will need.  For ~300 - 400 components and 3 - 4 node cluster, the NCM seems pretty good with 8GB of heap.  If you encounter heap issue still, you would need to increase the heap size and/or reduce the stat buffer size and/or frequency in the nifi.properties files (NCM and Nodes).

nifi.components.status.repository.buffer.size=360            (defaults is 1440)

nifi.components.status.snapshot.frequency=5 min          (default is 1)

This information is accurate as of NiFi 0.5.1 and HDF 1.1.2.