Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Nifi JVM Memory Utilization Continually Rising...

avatar
Contributor

Hey Guys,

We have a Nifi cluster that 5 days ago hung - the flow.xml.gz file got out of sync between the nodes due to prolonged connectivity issue. Ambari was also reporting a 98% JVM memory utilization (we have 32 Gigs).

Once we resync'ed the flow.xml.gz file and restarted the nodes, the JVM memory dropped back down to 5%. Since then, the JVM memory has slowly climbed back up to 82% - which Ambari is warning with a color-coded orange.

We don't have 'alot' of flows nor queued flowfiles (~300 flowfiles of less than 10k size), so I'm puzzled at the high JVM memory utilization. Any help diagnosing this issue would be greatly appreciated.

1 ACCEPTED SOLUTION

avatar
Master Mentor
@Sonny Chee

What version of HDF are you running? Is it HDF 3.1?

Is your NiFi secured and using Ranger for NiFi authorizations?

If so, you may have run into this identified bug:

https://issues.apache.org/jira/browse/NIFI-4925

There are currently no HDF releases with the fix included.

Thank you,

Matt

View solution in original post

4 REPLIES 4

avatar
Master Mentor
@Sonny Chee

What version of HDF are you running? Is it HDF 3.1?

Is your NiFi secured and using Ranger for NiFi authorizations?

If so, you may have run into this identified bug:

https://issues.apache.org/jira/browse/NIFI-4925

There are currently no HDF releases with the fix included.

Thank you,

Matt

avatar
Contributor

Thank you for your detailed response, Matt @Matt Clarke.

Yes, we are using HDF-3.1.0.0 and it is secured and using Ranger for NiFi authorizations.

I see that this issue is resolved in Nifi 1.6.0. Is the recommended short term fix to upgrade Nifi?

avatar
Master Mentor
@Sonny Chee

Apache NiFi 1.6 has not been released yet. It also will not include with it the Ranger plugin stuff that is included with the HDF NiFi releases versions.

If you have a support contract with Hortonworks for HDF, you could requests a hotfix build from them.

The HDF NiFi releases are never identical to any Apache NiFi release. Some processors are modified to support HDP better and some additional libraries are added (such as Ranger).

Just as a note, you can always think of the HDF releases as being build on Apache NiFi <version> plus more.

avatar
Contributor

Thank you for the clarification.