Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How much Transaction per Second Apache NiFi can hold and How much FlowFile it can transfer pararelly?

avatar
New Contributor

I wanna know how much Transaction per Second Apache NiFi can be done and how many FlowFile maximum it can transfer pararelly.

 

Thank you for your assisstance

  • Add to Phrasebook
     
    • No word lists for English -> Indonesian...
       
    • Create a new word list...
  • Copy
1 REPLY 1

avatar
Super Guru

@derisrayan Your question is impossible to answer without very detailed inspection of the following items:

 

  1. NiFi Cluster Size (# of nodes) and Spec of each Node (CPU/RAM/Disk)
  2. The size of the data processing per flowfile
  3. The number of pieces of the data arriving per execution of the flow

After the above, the configuration of the data flow for concurrency and parallelism is tuned to what NiFi Cluster performance capabilities.  This comes down to Total NiFi Nodes, Total Cores, the configuration and how many active threads the NiFi Cluster can handle.

 

With a nicely configured NiFi cluster (3+ nodes) with as much ram and cores as possible, the transactions will be quite impressive.  Scaling to 5-10-15+ nodes will increase this to an impressive production ready scale.

 

 

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  

 

Thanks,


Steven @ DFHZ