Ambari agent in HDP 2.2.0 seems to be taking about 22Gigs after running for a while at a customer site.
Is there a way to reduce or limit its memory usage
This is a known Ambari bug:
This is resolved in Ambari Agent 2.4.0
As a work around you can modify the main.py
1. Stop Ambari agent
2. Backup file
mv /usr/lib/python2.6/site-packages/ambari_agent/main.py /tmp/main.py.backup
3. Edit main.py under this path : /usr/lib/python2.6/site-packages/ambari_agent/
Add the following at the beginning of the file:
""" Subprocess in Python has race condition with enabling/disabling gc. Which may lead to turning off python garbage collector. This leads to a memory leak. This function monkey patches subprocess to fix the issue. !!! PLEASE NOTE THIS SHOULD BE CALLED BEFORE ANY OTHER INITIALIZATION was done to avoid already created links to subprocess or subprocess.gc or gc """
# monkey patching subprocess
import subprocess subprocess.gc.isenabled = lambda: True
# re-importing gc to have correct isenabled for non-subprocess contexts
import sys del sys.modules['gc']
import gc fix_subprocess_racecondition()
4. Start Ambari agent
Please do this on one host and monitor the memory usage. If memory usage looks OK, then replace main.py on all hosts.