Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)

SYMPTOMS: /tmp filling up causes multiple services to stop functioning.

ROOT CAUSE: The issue happens due to internal Smartsense bug ST-2551.

SOLUTION: Upgrade to Smartsense 1.3.1

WORKAROUND: To workaround this issue we need to manually modify two files related to Smartsense, so that the tmp files will not be generated in /tmp directory anymore

1. File : /usr/hdp/share/hst/hst-agent/lib/hst_agent/anonymize.py

Change from :

ANONYMIZATION_JAVA_COMMAND = "{0}" + os.sep + "bin" + os.sep + "java" +\ 
" -Dlog.file.name="+ ANONYMIZATION_LOG_FILE_NAME +\ " -cp {1} {2} {3}" 

Change to :

ANONYMIZATION_JAVA_COMMAND = "{0}" + os.sep + "bin" + os.sep + "java" +\ 
" -Djava.io.tmpdir=/grid/02/smartsense/hst-agent/data/tmp" +\ 
" -Dlog.file.name="+ ANONYMIZATION_LOG_FILE_NAME +\ 
" -cp {1} {2} {3}" 

Make sure the tmp dir value is same as this property agent.tmp_dir in hst-agent-conf.

2. File : /usr/sbin/hst-server.py

Change from :

SERVER_START_CMD = "{0}" + os.sep + "bin" + os.sep +\ "java -server -XX:NewRatio=3 "\ 
"-XX:+UseConcMarkSweepGC " +\ 
"-XX:-UseGCOverheadLimit -XX:CMSInitiatingOccupancyFraction=60 " +\ 
debug_options +\ 
" -Dlog.file.name="+ SERVER_LOG_FILE_NAME +" -Xms512m -Xmx2048m -cp {1}" + os.pathsep + "{2}" +\ 
" com.hortonworks.support.tools.server.SupportToolServer "\ 
">" + SERVER_OUT_FILE + " 2>&1 &" 

Change to :

SERVER_START_CMD = "{0}" + os.sep + "bin" + os.sep +\ 
"java -server -XX:NewRatio=3 "\ "-XX:+UseConcMarkSweepGC " +\ 
"-XX:-UseGCOverheadLimit -XX:CMSInitiatingOccupancyFraction=60 " +\ 
"-Djava.io.tmpdir=/var/lib/smartsense/hst-server/tmp " +\ 
debug_options +\ 
" -Dlog.file.name="+ SERVER_LOG_FILE_NAME +" -Xms512m -Xmx2048m -cp {1}" + os.pathsep + "{2}" +\ 
" com.hortonworks.support.tools.server.SupportToolServer "\ 
">" + SERVER_OUT_FILE + " 2>&1 &" 

Make sure the tmp dir value is same as this property server.tmp.dir in hst-server-conf.

3. After above changes, pease clean up existing .pyc files from both of the above directories, and restart Smartsense server and agents for changes to take effect.

431 Views
0 Kudos
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎12-24-2016 02:36 PM
Updated by:
 
Contributors
Top Kudoed Authors