@Chetankc  From a NiFi perspective there is not much guidance that can be given with such little information. What does "10 Billion Load" mean?  Is it the number if unique files being ingested to NiFi?  What is size average?  What is rate of ingest?   What is "15,000 process"?  Is this the number of NiFi processors added to the NiFi canvas?  What types of processors are being used?  Does your dataflow(s) do a lot of content modification? Have you done testing on throughput performance and done any performance tuning?  15,000 processors is a lot of execution scheduling against your CPU cores. In your load testing what was you CPU load average?  What was your memory impact?  You also have custom NiFi components.  Are you referring to these custom components as using many threads or the totality of the 15,000 components using a lot of threads? What does a lot of threads mean here? Are any of these long running threads or are they all millisecond thread executions? What kind of performance and throughput are you achieving now?   and onn what type of setup (how many nodes in your NiFi cluster, number of CPU cores, JVM Heap settings, type of disk, etc) currently?  Thank you, Matt ... View more