Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

ConsumeKafka_2_6 leaking lot of memory in apache nifi 1.25

avatar
Master Collaborator

Hello Expert,


I am using "ConsumeKafka_2_6 " processor to consume data in apache nifi 1.25.0.

In one of the case user has configured the processors with wrong/invalid kafka details.
And due to this nifi started consuming hell lot of memory and as we run nifi on k8,
pod got killed with OOM and was restarting every 10 mins.

We stopped each PG by PG to dig down and conclude that this processor is culprit.

This issue happened only twice as of now.
i.e it does not behave this all the time, may be with specific condition or so.

Any suggestion/solution for this issue is much appreciated.

 

Thanks in advance.

Mahendra

1 REPLY 1

avatar
Master Mentor

@hegdemahendra 

Did you take heap dumps to confirm which class was consuming the heap?
Any thread dump analysis when heap usage was growing?

What incorrect values were configured that you feel led to this component consuming large amounts of heap memory?

Not really finding any known issues of memory with this consumeKafka processor.  Any details you can provide (processor configuration, log exceptions while it was running with bad config, etc) may help.

Thank you,
Matt