Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Capacity planning for NiFi cluster

avatar

Is there any guide available for NiFi capacity planning ? What load characteristic should be looked at ?

- Number of concurrent processors running

- Total throughput

Any help is greatly appreciated.

1 ACCEPTED SOLUTION

avatar
Master Mentor
11 REPLIES 11

avatar

Thank You @mclark This is exactly what I was looking for.

avatar
Master Mentor

The NCM in a NiFi cluster typically needs more heap memory. The number of components (processors, input ports, output ports and relationships) x the number of nodes in the NiFi cluster on the graph will drive how much memory your NCM will need.  For ~300 - 400 components and 3 - 4 node cluster, the NCM seems pretty good with 8GB of heap.  If you encounter heap issue still, you would need to increase the heap size and/or reduce the stat buffer size and/or frequency in the nifi.properties files (NCM and Nodes).

nifi.components.status.repository.buffer.size=360            (defaults is 1440)

nifi.components.status.snapshot.frequency=5 min          (default is 1)

This information is accurate as of NiFi 0.5.1 and HDF 1.1.2.