Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi: Debugging high memory consumption

Highlighted

NiFi: Debugging high memory consumption

New Contributor

We've started using NiFi not long ago, and have a few flows running.

We're expriencing some high memory usage and are not sure how to debug it to see what's causing it.

During normal operations the java process is around ~8GB, then at seemingly random times memory shoots up to ~62GB (the Xmx parameter) in about ten minutes. After the spike up the memory remains at 62GB for hours. Only a restart makes it fall back to previous values.

Other than that, we don't seem to be having any errors - no out of memory exceptions, no large queues etc. nifi-app logs are pretty clear and don't have a single error or warning around the times of the spikes.

We've seen some people say to make sure G1GC is enabled, and it is enabled.

I'm not really sure how to proceed in trying to find what's consuming so much memory. Any ideas?

1 REPLY 1

Re: NiFi: Debugging high memory consumption

Super Guru

@Amir Formic

if you are using HDF 3.1 and Ranger for NiFi authorizations then There is ranger authourizer memory leak which is causing high memory usage in HDF-3.1 and issue got resolved in NiFi 1.6.0.

Please look into below jira

https://issues.apache.org/jira/browse/NIFI-4925

Community question related to high memory usage.

https://community.hortonworks.com/questions/183289/nifi-jvm-memory-utilization-continually-rising.ht...