Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Nifi memory continues to expand

avatar
New Contributor

I used a three-node nifi cluster, the nifi version is 1.16.3, the hardware is 8core 32G memory, and the solid-state high-speed hard disk is 2T. OS is CentOS7.9, ARM64 hardware architecture. The initial configuration of nifi is xms12g and xmx12G(bootstrip.conf).

Native installation, docker is not used, and only nifi installed on all thoese machines, using integrated zookeeper.

 

Run 20 workflow(task) everyday from 00:00 to 03:00, and the total data size is 1.2G. Collect csv documents to the greenplum database.

those tasks consume little resources. Even if the memory is adjusted to 6G, 20 tasks run at the same time, the memory consumption is about 30%, and it will run out in half an hour.

 

My problem now is that the memory usage of nifi is increasing every day, 0.2G per day, and all three nodes are like this. Then the memory is slowly full and then the machine is dead. This procedure is about a month(when the memory is set to 12G.).

That is to say, I need to restart the cluster every month. I use a native processor and workflow.

I can't locate the problem. Who can help me? I may have any descriptions. Please feel to let me know,thanks.

I have made the following attempts:

  1. I set the initial memory to 18G or 6G, and the speed of workflow processing has not changed. The difference is that, after setting it to 18G, it will freeze for a shorter time.

  2. I used openjre1.8, and I tried to upgrade it to 11, but it was useless.

  3. i add the following configuration, and is also useless:

java.arg.7=-XX:ReservedCodeCacheSize=256m
java.arg.8=-XX:CodeCacheMinimumFreeSpace=10m
java.arg.9=-XX:+UseCodeCacheFlushing

 

0 REPLIES 0