Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NIFI: How to turn on GC logging for NiFi?

avatar

Hi

We are having difficulties with cluster nodes periodically becoming disconnected from the cluster. The kind of errors we are seeing in the logs are the same as the ones mentioned in this question: https://community.hortonworks.com/questions/93694/nifi-10-node-suddenly-disconnected-and-connected.h...

The suggestion in that question is that the problem is related to problems communication with Zookeeper (which is running fine in our system) and could possibly be related to Garbage Collection within NiFi causing an interruption to processing.

I would therefore like to gather some evidence as to how long GC is taking, but I can't see any GC information in the NiFi logs.

Can you tell me if it's possible to switch on logging of garbage collection within NiFi, and if not, suggest a way of monitoring GC, bearing in mind that it would not be possible for me to download and install some third-party monitoring tool on our system.

Thanks

Richard

1 ACCEPTED SOLUTION

avatar
Super Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
5 REPLIES 5

avatar
Super Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar

Thank you. I've tested this on our Dev system and it looks like it's exactly what we need.

Regards

Richard

avatar
Contributor

Thank you for your solution. It works wonders.

In the meantime I don't think there is a default housekeeping process.

Do you know how to control the rollover of the GC logs?

 

Thank you in advance

avatar
Super Mentor

@DavidR 

 

Typically you would only enable GC debug logging to investigate and issue and not leave these properties in all the time.

If you intend on leaving these in all the time, i suggest removing the following line:

 

java.arg.23=-Xloggc:<file>

 

 

With no log file defined the GC logs will go directly in to the nifi-bootstrap.log.

You can then easily control/configure log rotation and retention for the NiFi bootstrap log in the logback.xml file found in the NiFi conf directory.

 

Thanks,

Matt 

avatar
Contributor

@MattWho 

Thank you for your input. It works and It was invaluable.