question NIFI: How to turn on GC logging for NiFi? in Archives of Support Questions (Read Only)

NIFI: How to turn on GC logging for NiFi?

richard_d_corfi — Wed, 31 May 2017 20:31:02 GMT

We are having difficulties with cluster nodes periodically becoming disconnected from the cluster. The kind of errors we are seeing in the logs are the same as the ones mentioned in this question: https://community.hortonworks.com/questions/93694/nifi-10-node-suddenly-disconnected-and-connected.html

The suggestion in that question is that the problem is related to problems communication with Zookeeper (which is running fine in our system) and could possibly be related to Garbage Collection within NiFi causing an interruption to processing.

I would therefore like to gather some evidence as to how long GC is taking, but I can't see any GC information in the NiFi logs.

Can you tell me if it's possible to switch on logging of garbage collection within NiFi, and if not, suggest a way of monitoring GC, bearing in mind that it would not be possible for me to download and install some third-party monitoring tool on our system.

Thanks

Richard

Re: NIFI: How to turn on GC logging for NiFi?

MattWho — Wed, 31 May 2017 20:44:58 GMT

@Richard Corfield

You can get GC output logs out of NiFi by adding the following lines to NiFi's bootstrap.conf file:

java.arg.20=-XX:+PrintGCDetails

java.arg.21=-XX:+PrintGCTimeStamps

java.arg.22=-XX:+PrintGCDateStamps

java.arg.23=-Xloggc:<file>

The last entry allows you to specific a separate log file for this output to be written in to rather then stdout.

Thanks,

Matt

Re: NIFI: How to turn on GC logging for NiFi?

richard_d_corfi — Wed, 31 May 2017 21:30:35 GMT

Thank you. I've tested this on our Dev system and it looks like it's exactly what we need.

Regards

Richard

Re: NIFI: How to turn on GC logging for NiFi?

DavidR — Mon, 28 Oct 2019 16:24:48 GMT

Thank you for your solution. It works wonders.

In the meantime I don't think there is a default housekeeping process.

Do you know how to control the rollover of the GC logs?

Thank you in advance

Re: NIFI: How to turn on GC logging for NiFi?

MattWho — Mon, 28 Oct 2019 19:05:23 GMT

@DavidR

Typically you would only enable GC debug logging to investigate and issue and not leave these properties in all the time.

If you intend on leaving these in all the time, i suggest removing the following line:

java.arg.23=-Xloggc:<file>

With no log file defined the GC logs will go directly in to the nifi-bootstrap.log.

You can then easily control/configure log rotation and retention for the NiFi bootstrap log in the logback.xml file found in the NiFi conf directory.

Thanks,

Matt

Re: NIFI: How to turn on GC logging for NiFi?

DavidR — Tue, 29 Oct 2019 11:21:18 GMT

@MattWho

Thank you for your input. It works and It was invaluable.