Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NodeManger doesn’t log the Memory usage message

avatar
Rising Star

Hello,

I use yarn (base on HDP3.1.5.0-152),when I view the nodemanager log,I found the log doesn't note the log likes:

INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 28532 for container-id container_e51_1637148502450_21618293_01_000024: 4.1 GB of 4.5 GB physical memory used; 6.3 GB of 9.4 GB virtual memory used

I don't remember there are some conf in yarn-site.xml can control this.

Any one help?

1 ACCEPTED SOLUTION

avatar
Contributor

Since its moved from INFO level you can set the NM log level to DEBUG mode to get the container metrics on nm log. But that will be applied globally hence you can try to enable only on  ContainersMonitorImpl.
Append to nodemanager log4j
 log4j.logger.org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl.audit=DEBUG

 

Make sure to mark the answer as the accepted solution. If it resolves your issue !

View solution in original post

5 REPLIES 5

avatar
Contributor

Currently on HDP3.1.5.0-152 logging of container memory usage is not possible. Because it got removed from the source code of HDP3. 
Make sure to mark the answer as the accepted solution. If it resolves your issue !

avatar
Rising Star

Thanks,too sad.

So,Is there another way to do that?

I want to track the resource status of the container execution

avatar
Contributor

Since its moved from INFO level you can set the NM log level to DEBUG mode to get the container metrics on nm log. But that will be applied globally hence you can try to enable only on  ContainersMonitorImpl.
Append to nodemanager log4j
 log4j.logger.org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl.audit=DEBUG

 

Make sure to mark the answer as the accepted solution. If it resolves your issue !

avatar
Rising Star

Thanks!It works!

Meepoljd_0-1642493532225.png

 

avatar
Community Manager

Hi @Meepoljd, glad to know that your issue was fixed. Can you please accept @Amithsha's response as a solution? It will make it easier for others to find the answer in the future. 



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community: