Support Questions

Find answers, ask questions, and share your expertise

ConsumeKafka_2_6 leaking lot of memory in apache nifi 1.25

avatar
Master Collaborator

Hello Expert,


I am using "ConsumeKafka_2_6 " processor to consume data in apache nifi 1.25.0.

In one of the case user has configured the processors with wrong/invalid kafka details.
And due to this nifi started consuming hell lot of memory and as we run nifi on k8,
pod got killed with OOM and was restarting every 10 mins.

We stopped each PG by PG to dig down and conclude that this processor is culprit.

This issue happened only twice as of now.
i.e it does not behave this all the time, may be with specific condition or so.

Any suggestion/solution for this issue is much appreciated.

 

Thanks in advance.

Mahendra

1 REPLY 1

avatar
Master Mentor

@hegdemahendra 

Did you take heap dumps to confirm which class was consuming the heap?
Any thread dump analysis when heap usage was growing?

What incorrect values were configured that you feel led to this component consuming large amounts of heap memory?

Not really finding any known issues of memory with this consumeKafka processor.  Any details you can provide (processor configuration, log exceptions while it was running with bad config, etc) may help.

Thank you,
Matt