NodeManger doesn’t log the Memory usage message

Question

Hello,I use yarn (base on HDP3.1.5.0-152),when I view the nodemanager log,I found the log doesn't note the log likes:INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 28532 for container-id container_e51_1637148502450_21618293_01_000024: 4.1 GB of 4.5 GB physical memory used; 6.3 GB of 9.4 GB virtual memory usedI don't remember there are some conf in yarn-site.xml can control this.Any one help?

Amithsha · Accepted Answer

Since its moved from INFO level you can set the NM log level to DEBUG mode to get the container metrics on nm log. But that will be applied globally hence you can try to enable only on ContainersMonitorImpl.
Append to nodemanager log4j
log4j.logger.org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl.audit=DEBUG

Make sure to mark the answer as the accepted solution. If it resolves your issue !

Amithsha · Answer

Currently on HDP3.1.5.0-152 logging of container memory usage is not possible. Because it got removed from the source code of HDP3.
Make sure to mark the answer as the accepted solution. If it resolves your issue !

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.