Support Questions
Find answers, ask questions, and share your expertise

NodeManger doesn’t log the Memory usage message

Explorer

Hello,

I use yarn (base on HDP3.1.5.0-152),when I view the nodemanager log,I found the log doesn't note the log likes:

INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 28532 for container-id container_e51_1637148502450_21618293_01_000024: 4.1 GB of 4.5 GB physical memory used; 6.3 GB of 9.4 GB virtual memory used

I don't remember there are some conf in yarn-site.xml can control this.

Any one help?

1 ACCEPTED SOLUTION

Cloudera Employee

Since its moved from INFO level you can set the NM log level to DEBUG mode to get the container metrics on nm log. But that will be applied globally hence you can try to enable only on  ContainersMonitorImpl.
Append to nodemanager log4j
 log4j.logger.org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl.audit=DEBUG

 

Make sure to mark the answer as the accepted solution. If it resolves your issue !

View solution in original post

5 REPLIES 5

Cloudera Employee

Currently on HDP3.1.5.0-152 logging of container memory usage is not possible. Because it got removed from the source code of HDP3. 
Make sure to mark the answer as the accepted solution. If it resolves your issue !

Explorer

Thanks,too sad.

So,Is there another way to do that?

I want to track the resource status of the container execution

Cloudera Employee

Since its moved from INFO level you can set the NM log level to DEBUG mode to get the container metrics on nm log. But that will be applied globally hence you can try to enable only on  ContainersMonitorImpl.
Append to nodemanager log4j
 log4j.logger.org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl.audit=DEBUG

 

Make sure to mark the answer as the accepted solution. If it resolves your issue !

Explorer

Thanks!It works!

Meepoljd_0-1642493532225.png

 

Community Manager

Hi @Meepoljd, glad to know that your issue was fixed. Can you please accept @Amithsha's response as a solution? It will make it easier for others to find the answer in the future. 



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:
; ;