Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Is there any problem with the JVM of NiFi, is there any failover/ practice

Solved Go to solution
Highlighted

Is there any problem with the JVM of NiFi, is there any failover/ practice

Contributor

Hi All,

Thanks a lot to this community.

I am working on building data flows, and i have 40 to 50 data flows to be configured. some of them are like 15000 events per second.

If I have some of the data flows configured and then a very high event per second data source is being configured, and some how fills up the JVM, everything crashes because of the disk space,

or sudden event burst occurs

Does it mean we will have to proactively watch for any space issues such as JVM heap size, content repo and provenance repo. etc

Thanks

Dheiru

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Is there any problem with the JVM of NiFi, is there any failover/ practice

Hi @dhieru singh

You need to do two things:

  • First, you need a good capacity planing to evaluate the required infrastructure that can handle your data flows. Consider the worst case scenario to have room for improvement and the capacity to manage bursts. There are several resources out there that can help you

https://community.hortonworks.com/articles/135337/nifi-sizing-guide-deployment-best-practices.html

https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.0/bk_command-line-installation/content/hdf_is...

1 REPLY 1

Re: Is there any problem with the JVM of NiFi, is there any failover/ practice

Hi @dhieru singh

You need to do two things:

  • First, you need a good capacity planing to evaluate the required infrastructure that can handle your data flows. Consider the worst case scenario to have room for improvement and the capacity to manage bursts. There are several resources out there that can help you

https://community.hortonworks.com/articles/135337/nifi-sizing-guide-deployment-best-practices.html

https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.0/bk_command-line-installation/content/hdf_is...

Don't have an account?
Coming from Hortonworks? Activate your account here