Smartsense agent dying

[#1]

When using smartsense 1.2 or below in conjunction with OpenJDK, you get the following error upon startup. It's a none issue which will be resolved in the next smart sense version.

Traceback (most recent call last):
  File "/usr/sbin/hst-agent.py", line 420, in <module> main(sys.argv)
  File "/usr/sbin/hst-agent.py", line 397, in main setup(options)
  File "/usr/sbin/hst-agent.py", line 323, in setup server_hostname = get_server_hostname(server, tries, try_sleep, options.quiet)
  File "/usr/sbin/hst-agent.py", line 107, in get_server_hostname hostname = validate_server_hostname(default_hostname, tries, try_sleep)
  File "/usr/sbin/hst-agent.py", line 125, in validate_server_hostname elif not register_agent(server_hostname):
  File "/usr/sbin/hst-agent.py", line 143, in register_agent if not server_api.register_agent(agent_version):
  File "/usr/hdp/share/hst/hst-agent/lib/hst_agent/ServerAPI.py", line 104, in register_agent content = self.call(request)
  File "/usr/hdp/share/hst/hst-agent/lib/hst_agent/ServerAPI.py", line 52, in call self.cachedconnect = security.CachedHTTPSConnection(self.config)
  File "/usr/hdp/share/hst/hst-agent/lib/hst_agent/security.py", line 111, in __init__ self.connect()
  File "/usr/hdp/share/hst/hst-agent/lib/hst_agent/security.py", line 116, in connect self.httpsconn.connect()
  File "/usr/hdp/share/hst/hst-agent/lib/hst_agent/security.py", line 87, in connect raise err
ssl.SSLError: [Errno 8] _ssl.c:492: EOF occurred in violation of protocol

To fix this issue, you will need to modify the SSL Digest from md5 to sha256.

Here are the steps required to do it.

• From Ambari stop the SmartSense service ( all components )
• Backup the old server keys on the HST server host

cp -rp /var/lib/smartsense/hst-server/keys /var/lib/smartsense/hst-server/keys.backup

• Clean out the old keys on the HST server host

rm -f /var/lib/smartsense/hst-server/keys/ca.key
rm -f /var/lib/smartsense/hst-server/keys/*.csr
rm -f /var/lib/smartsense/hst-server/keys/*.crt
rm -rf /var/lib/smartsense/hst-server/keys/db/*
mkdir /var/lib/smartsense/hst-server/keys/db/newcerts
touch /var/lib/smartsense/hst-server/keys/db/index.txt
echo 01 > /var/lib/smartsense/hst-server/keys/db/serial

• Modify default digest on HST server host

Edit file /var/lib/smartsense/hst-server/keys/ca.config 
change line "default_md = md5"  to "default_md = sha256"

• Clean out the old keys on each HST Agent hosts.

rm -f /var/lib/smartsense/hst-agent/keys/*

• If using HST Gateway, on HST gateway stop the service and remove certs

hst gateway stop
rm -f /var/lib/smartsense/hst-gateway/keys/ca.key
rm -f /var/lib/smartsense/hst-gateway/keys/*.csr
rm -f /var/lib/smartsense/hst-gateway/keys/*.crt
rm -rf /var/lib/smartsense/hst-gateway/keys/db/*
mkdir /var/lib/smartsense/hst-gateway/keys/db/newcerts
touch /var/lib/smartsense/hst-gateway/keys/db/index.txt
echo 01 > /var/lib/smartsense/hst-gateway/keys/db/serial

• If using HST Gateway, modify default digest on HST gateway host

Edit file /var/lib/smartsense/hst-gateway/keys/ca.config 
change line "default_md = md5"  to "default_md = sha256"

• If using HST Gateway, on HST server remove old certs

rm -f /var/lib/smartsense/hst-gateway-client/keys

• If using HST Gateway, on HST Gateway restart service

 hst gateway start

• Restart SmartSense service from Ambari ( all components ) and verify both Ambari SmartSense service and SmartSense view shows correct number of agents registered.

[#2]

This occurs on hosts with following JDK versions or newer:

JDK Family	Versions
Oracle	1.8.0_71
Oracle	1.7.0_95
Oracle	1.6.0_111
OpenJDK	1.7.0_45
OpenJDK	1.8.0_40

It is also recommended to upgrade to SmartSense 1.2.1+ while applying these changes.

[#3]

I had smartsense-hst-1.1.0 packages installed on CentOS 6 (with Oracle JDK 1.8.0_73) affected by this issue.

To follow these instructions I had to replace

/var/lib/smartsense/

with

/usr/hdp/share/hst/

in the paths given above.

E.g instead of

rm -f /var/lib/smartsense/hst-gateway/keys/*.crt

I used

rm -f /usr/hdp/share/hst/hst-gateway/keys/*.crt

Thanks for posting the solution.