Heavy CPU usage by NiFi Java process

[#1]

Hi All,

I have a simple NiFi Dataflow on my local Windows machine, but NiFi's Java process utilization on my machine is spiking high; I was not sure what's causing the CPU to spike, so I closed the NiFi instance (aborted the running run-nifi.bat batch file and killed the Java Process in Task Manager) and restarted NiFi, and found no change, CPU still spiking; this spiking is without Starting any of the NiFi Processors; also, uncommented these 3 properties in bootstrap.conf, but still no help;

java.arg.7=-XX:ReservedCodeCacheSize=256m

java.arg.8=-XX:CodeCacheFlushingMinimumFreeSpace=10m

java.arg.9=-XX:+UseCodeCacheFlushing

I then started to check system diagnostics, which showed that both "FlowFile repository storage usage" and "Content repository storage usage" are at 86% (271 GB of 315 GB); I don't know what those numbers mean, but I'm puzzled; I don't even have that much free space (271 GB) on my machine, what do these numbers represent; and how do I flush these and get a clean start, thinking that doing so would help the CPU spiking issue.

Any help is appreciated.

Thanks,

Raj

[#2 · Accepted solution]

Hi Raj,

Have you flows running with NiFi? If processors were running when stopping NiFi, they will be running when NiFi starts.

If yes, could you tell us what processors are running? Also, could you have a look into the logs (nifi-app.log) to copy/paste here anything that could be relevant?

Regarding repositorie: the Content Repository holds the content for all the FlowFiles in the system, and the FlowFile repository keeps track of the attributes and current state of each FlowFile in the system. (I'd recommend having a look here: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html)

[#3]

is there anything in the logs?

[#4]

When I experienced this issue, all NiFi Processors are stopped; the only thing was there was one connection that was queued up; trying to empty the queue was not working (a progress bar would show up, but it would just stay at 0% and not progress)

Checked the nifi-app.log file; "java.lang.OutOfMemoryError: GC overhead limit exceeded" is in the log quite a lot.

Since I'm running NiFi on my local laptop, I unzipped/installed NiFi again in any another folder and exported/imported the dataflow as a template from the old folder to the new folder; that seems to have resolved the issue I was facing.

[#5]

How many flow files in the queue?

It sounds like you used a processor generating large amounts of FlowFiles and your computer was not able to handle so much. You may want to consider back pressure features, or changing the scheduling of processors.

[#6]

I am reading flat files from a Unix directory, there are about 8 million files, the files are small, 1Kb-2Kb; but, that may have been the problem; thanks for the suggestion, I will look into back pressure and scheduling; thanks Pierre

[#7]

that's strange, what JDK is on my machine

[#8]

It's JDK 1.8

[#9]

I had something like this happen on my 16G macbook pro when I switched from 0.4.1 to 0.6.0 and reused the repository directories. It was just an experimental instance, so I abandoned the original repos and started from scratch, except for importing a few flows. I have not had the problem since. The odd thing in my case is that it was fine in 0.4.1 but not in 0.6.0.