What does NiFi use non-heap memory for?

[#1]

I'm trying to make sense of the below screenshot. Questions:

1. What is non-heap memory used for? I couldn't find any reference in the admin guide.
2. Does -1 mean uncapped memory usage? What is a correlation between Total and Max here?
3. Should I be concerned there's only ~11MB left of free non-heap memory? Should we bump it? How?

[#2 · Accepted solution]

These are just standard structures of the JVM. For example this is where things like the permgen live. The permgen is where in Java 7 and older class definitions and other things are loaded. We probably should just remove max for non-Heap as I'm not sure the idea of a max there makes sense.

Now, regarding the fact that free space is such a small size compared to the total is something worth consideration. In the conf/bootstrap.conf you will see two really important lines which are commented out by default:

#java.arg.11=-XX:PermSize=128M
#java.arg.12=-XX:MaxPermSize=128M

We do often recommend uncommenting those lines as the default size of the permgen is insufficient for how many classes NiFi can load at startup especially given its isolated classloader design. Simply uncommenting those lines will do the trick. Depending on the amount of classes that end up getting loaded and/or loading that may occur dynamically you may need more or less space allocated. If the permgen runs out of space entirely system performance will suffer so best to avoid this. We need to update the install guide to call this and all bootstrap properties out. We don't want to have these set by default because in Java 8 the permgen was restructured and is now part of the heap.

Thanks!

[#3]

Thanks, Joe. I'm aware of Java 8 dropping those settings, wherever they decided to have permgen be now, good choice there with defaults in bootstrap.

I now realized that I got thrown off by off-heap vs non-heap, was thinking of the off-heap memory for this reason. I realize there's no off-heap memory used in the product.