Use CA Signed certs to enable two-way SSL between ambari-server and ambari-agent

[#1]

Steps to configure 2way SSL between Ambari-server and Ambari-agent by using custom certs.

Here I have used CA Signed certs at server side and Agent certs are generated dynamically. if you are planning to use CA Signed cert at Agents side then for every Agent install you may have to copy the certs and do manual work.

1. Make sure to have fresh keys folder. (if you do not have one, you can copy the folder from one of the fresh install machine or do following).
   - Delete all the crt and csr files that starts with hostname at /var/lib/ambari-server/keys.
   - Empty /var/lib/ambari-server/keys/db/index.txt file
   - Delete any certs under /var/lib/ambari-server/keys/db/newcerts/

2.  - Copy your own Signed Certificate, key files /var/lib/ambari-server/keys/
     Ex: certificate name is - ca-cust.crt, ca-cust.key

3. Create PKCS keystore file from your cert and key files.
    Ex:openssl pkcs12 -export -inkey /tmpr/keys/ca.key  -in ca-cust.crt   -out /tmp/keys/keystore.p12
     -password pass:bigdata -passin pass:bigdata
    Note: replace passwords with appropriate

4. Create pass-cust.txt with appropriate password that is been provided in step3 for keystore.
   Ex: echo "bigdata" > pass-cust.txt

5. Configure your ambari.properties with appropriate cert, keys, keystore file names.

        security.server.cert_name=ca-cust.crt
	security.server.key_name=ca-cust.key
	security.server.keystore_name=keystore-cust.p12
	security.server.truststore_name=keystore-cust.p12
	security.server.crt_pass_file=pass-cust.txt
	security.server.two_way_ssl=true

6. remove any existing certs in all the Agent hosts at  /var/lib/ambari-agent/keys/

7. start ambari-server and ambari-agent logs

Note1: look out for SSL errors in ambari-server logs during startup. this is tried in Ambari2.4.x

Have tried with 2.6.x and it works fine too.

Note2: Currently there is a BUG https://issues.apache.org/jira/browse/AMBARI-23920 in the product - please follow the workaround mentioned.

[#2]

I tried your steps above, but the ambari-server generates certificates on the agent nodes. To give some context, I have a single certificate with multiple 'subject alternative names' for all the nodes in the cluster. I put that 1 certificate under the /var/lib/ambari-agent/keys folder on all the agents and as soon as I restart Ambari-server, it still does not pick up my '.crt' instead it begins generating the .key, .csr and .crt.

My goal is to use the .crt I have to be used by the agent and server on all the nodes for the two_way_ssl functionality.

Please advise.

[#3]

***Update - I switched back to using Ambari's generated certificate for agents and the server, as I was getting SSL errors related to having the certs not signed by the same 'CA'. Is this because I was using the self-signed certificate for testing locally? I havent' tried this with the CA signed multiple SAN certificate.

Also, while comparing the 'Subject Name' on the certificate generated by the Ambari server and the multiple 'Subject Alternative Name' certificate I intended to use originally, the 'Subject name's would have caused discrepancy. Looks like ambari server looks for the node name in the Subject line, but in the SAN certificate I have, the names of the nodes as part of the 'V3 extensions' in the certificate.

If you have any suggestions for this scenario, please post.

Thanks.

[#4]

@Anitha R

sorry for late reply - am not available currently.

these steps are only for CA signed certs at server side and client certs are generated dynamically.

[#5]

Thank you for the reply. I switched to using ambari generated certs for all instead. The instructions were helpful. Thank you.