Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
Super Mentor

In this example we will see how to use a self-signed certificate for initial trials to setup ambari console access over HTTPS. Remember that the self signed certificates are not suitable for production environments. So if your organization does not provide you with an SSL server certificate, you must request a new certificate that is signed by a CA.

Users can use several methods to obtain a new signed certificate. For example, users can use Microsoft Internet Information Services (IIS) Manager to request an SSL server certificate from a CA. For testing purposes, you can obtain a free temporary certificate based on an untrusted root from many CAs. To know more about it refer to : https://en.wikipedia.org/wiki/Certificate_authority.

This article is divided into 4 sections.

Section-1). Generating keys self signed certificates.

Section-2). Configuring Ambari Server to use the certificates.

Section-3). Common troubleshooting steps.

Section-4). Setting up the Ambari Server truststore.

.

Generating keys self signed certificates.

Make sure that "openssl" is installed.

Step-1). Set the ambari host name as an environment variable "AMBARI_SERVER_HOSTNAME" which we will later use to create the certificate name like "ambari1.example.com.key". Users can choose any name for their certificates/keys not necessarily the hostname.

mkdir /etc/ambari-server/certs 
cd /etc/ambari-server/certs/ 
export AMBARI_SERVER_HOSTNAME=ambari1.example.com

Step-2). Use the "genrsa" command which will generate an RSA private key. 2048 is the size of the private key to generate in bits. The default is 512. This must be the last option specified.

cd /etc/ambari-server/certs/
openssl genrsa -passout pass:hadoop -out $AMBARI_SERVER_HOSTNAME.key 2048

Output should be something like following:

[root@ambari1 ~]# openssl genrsa -passout pass:hadoop -out $AMBARI_SERVER_HOSTNAME.key 2048
 Generating RSA private key, 2048 bit long modulus
 ..+++
 ........+++

Step-3). Use the "req" command which primarily creates and processes certificate requests in PKCS#10 format. This option can also be used to create self signed certificates for use as root CAs for example.

openssl req -new -key $AMBARI_SERVER_HOSTNAME.key -out $AMBARI_SERVER_HOSTNAME.csr -subj "/C=IN/ST=Karnataka/L=Bangalore/O=hwx/CN=$AMBARI_SERVER_HOSTNAME"

Step-4). Generating private key "$AMBARI_SERVER_HOSTNAME.crt" using the option "-out" as following:

openssl x509 -req -days 365 -in $AMBARI_SERVER_HOSTNAME.csr -signkey $AMBARI_SERVER_HOSTNAME.key -out $AMBARI_SERVER_HOSTNAME.crt

Now in the "/etc/ambari-server/certs" directory users should see the following files:

[root@ambari1 certs]# cd /etc/ambari-server/certs
[root@ambari1 certs]# ls -lart
-rw-r--r--  1 root root 1675 Jun 15 00:01 ambari1.example.com.key
-rw-r--r--  1 root root  997 Jun 15 00:02 ambari1.example.com.csr
-rw-r--r--  1 root root 1184 Jun 15 00:02 ambari1.example.com.crt

.

Configuring Ambari Server to use the certificates.

Step-5). Use the [ambari-server setup-security] option to configure the certificates.

[root@ambari1 certs]# ambari-server setup-security
 Using python  /usr/bin/python
 Security setup options...
 ===========================================================================
 Choose one of the following options:
   [1] Enable HTTPS for Ambari server.
   [2] Encrypt passwords stored in ambari.properties file.
   [3] Setup Ambari kerberos JAAS configuration.
   [4] Setup truststore.
   [5] Import certificate to truststore.
 ===========================================================================
 Enter choice, (1-5): 1
 Do you want to configure HTTPS [y/n] (y)? y
 SSL port [8443] ? 8443
 Enter path to Certificate: /etc/ambari-server/certs/ambari1.example.com.crt
 Enter path to Private Key: /etc/ambari-server/certs/ambari1.example.com.key
 Please enter password for Private Key: hadoop
 Importing and saving Certificate...done.

Ambari server URL changed. To make use of the Tez View in Ambari please update the property tez.tez-ui.history-url.base in tez-site
 Adjusting ambari-server permissions and ownership...
 NOTE: Restart Ambari Server to apply changes ("ambari-server restart|stop+start")

Step-6). Restart the ambari server and then try accessing the following URL:

ambari-server restart

URL: https://ambari1.example.com:8443/#/main/dashboard/metrics

Step-7). As we are using untrusted Self signed certificates hence we will see the following message in the web browser. In this case it is normal hence click on the "Proceed to ambari1.example.com (unsafe)"

5025-err-cert-authority-invalid-warning.png

After proceeding further we will see the following console:

5033-ambari-with-https.png

.

Common troubleshooting steps?

Step-9). In order to troubleshoot any SSL related issue users can enable the "javax.net.debug" in the "/var/lib/ambari-server/ambari-env.sh" file by adding the following line in this file.

export AMBARI_JVM_ARGS=$AMBARI_JVM_ARGS' -Djavax.net.debug=ssl'

Now restart the Ambari Server. For more detailed debugging users can also use "-Djavax.net.debug=all"

Step-10). Due to the above debug flag now we can see the following kind of logging in the "ambari-server.out" file.

NOTE: This is "/var/log/ambari-server/ambari-server.out" STDOUT file and not the log file "/var/log/ambari-server/ambari-server.log".

INFO: Initiating Jersey application, version 'Jersey: 1.18 11/22/2013 01:21 AM'
***
found key for : 1
chain [0] = [
[
Version: V1
Subject: CN=ambari1.example.com, O=hwx, L=Bangalore, ST=Karnataka, C=IN
Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5

Key: Sun RSA public key, 2048 bits
modulus: 22175988901159630311396514667235749078146521163847992788959263233423755273156821620769244786460484287586602112399844584928245594827889197162513174308816812595461796535334661659155198218017182049796110412714443009008738097645038580643845113158858593395477455683493690692652792939276214070135191952868572717664244815115851111728266217307471648754541658144069497483918718341865043212851552819598807666038781460770942617500564581986900865648063480231718632571176573148407333265113899006488650860419740994935408185496217967665765639595473969860865574492100353172445226695292868455012371476694881518611913455083475879522469
public exponent: 65537
Validity: [From: Wed Jun 15 00:02:13 PDT 2016,
To: Thu Jun 15 00:02:13 PDT 2017]
Issuer: CN=ambari1.example.com, O=hwx, L=Bangalore, ST=Karnataka, C=IN
SerialNumber: [ ffeef037 fde4559f]

]
Algorithm: [SHA1withRSA]
Signature:
0000: 5F CF 38 F6 33 5D DE B9 2F 49 61 DD 08 AA EA 85 _.8.3]../Ia.....
0010: 91 D9 89 07 80 B6 91 61 17 76 42 F6 3F A8 D3 27 .......a.vB.?..'
0020: C0 A1 A2 1D B3 2C DD 90 AA 4D B2 E7 B7 55 89 0C .....,...M...U..
0030: C8 48 F9 6B 05 5E 9C C9 25 B8 72 49 DB 4A E0 C0 .H.k.^..%.rI.J..
0040: 32 F1 AB EF 4C C2 03 28 B7 C1 CB 3F 55 86 11 DB 2...L..(...?U...
0050: 72 0C AF AA D0 75 0F 85 4D 5B 90 C1 E3 2F A1 70 r....u..M[.../.p
0060: 58 57 57 5F C3 C5 64 25 A9 A9 C5 B1 11 FA 08 68 XWW_..d%.......h
0070: 4D 27 C7 69 CB E9 DE 0F 47 60 2E BE 3D 76 E6 51 M'.i....G`..=v.Q
0080: A5 04 4B 7F 81 83 F7 01 A8 A5 EE 1D E1 FE 20 8F ..K........... .
0090: 06 B0 3B 07 4A ED A7 BC F4 8F 2A 41 85 B6 C5 78 ..;.J.....*A...x
00A0: 91 FA 0F F8 A3 BE 75 77 12 9B 10 8E C2 B4 20 B3 ......uw...... .
00B0: 8D FB E3 71 80 BD B9 17 65 40 0C 01 11 3C 3D BB ...q....e@...<=.
00C0: 19 78 A8 D4 C7 28 DB 8B B6 14 5D 9D 8B B5 50 06 .x...(....]...P.
00D0: E4 D7 10 9A 63 29 2F 7A F8 91 11 7F A0 C2 78 8B ....c)/z......x.
00E0: 21 B1 AD 61 2F E4 E1 6F 84 74 72 7A 40 23 C5 EF !..a/..o.trz@#..
00F0: 4C 98 23 11 21 5A 1E 5B DA 2F 55 31 41 29 14 C7 L.#.!Z.[./U1A)..

]
***
adding as trusted cert:
Subject: CN=ambari1.example.com, O=hwx, L=Bangalore, ST=Karnataka, C=IN
Issuer: CN=ambari1.example.com, O=hwx, L=Bangalore, ST=Karnataka, C=IN
Algorithm: RSA; Serial number: 0xffeef037fde4559f
Valid from Wed Jun 15 00:02:13 PDT 2016 until Thu Jun 15 00:02:13 PDT 2017

trigger seeding of SecureRandom
done seeding SecureRandom
Using SSLEngineImpl.
Using SSLEngineImpl.
Jun 15, 2016 12:27:36 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ambari.server.agent.rest
org.apache.ambari.server.api
Using SSLEngineImpl.
Allow unsafe renegotiation: false

With the debug we can see the complete SSL communication detail in the out file and can find any failure in the communication. Also we can get other details like "unsupported cipher suite", etc.

Step-11). Users can also disable some weak ciphers by editing the "/etc/ambari-server/conf/ambari.properties" file by editing the following properties. For multiple ciphers users can separate each cipher using a vertical bar |.

security.server.disabled.ciphers=TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA

Step-12). To disable specific protocols like (SSL, SSLv2, SSLv3) , users can optionally add a list of the following format to ambari.properties.

security.server.disabled.protocols=SSL|SSLv2|SSLv3

.

Setting up Ambari Server truststore

What is truststore ? In a SSL handshake the purpose of trustStore is to verify credentials and the purpose of keyStore is to provide credential. keyStore in Java stores private key and certificates corresponding to their public keys and require if you are SSL Server or SSL requires client authentication. Users can use the "$JAVA_HOME/bin/keytool" to create the keystores/truststores.

If we want the SSL communication between Ambari and 3rd party services like "Hive View" then may need to configure the truststore in Ambari Server. For example if we configure the Ambari to run on HTTPs then if we navigate to the Ambari "Hive View" tab then we may find the following kind of error :

https://ambari1.example.com:8443/#/main/views/HIVE/1.0.0/AUTO_HIVE_INSTANCE

5034-hiveview-truststore-issue.png

Also on the ambari server logs we may see the same error as following "Truststore path or password is not set"

15 Jun 2016 05:30:14,861 ERROR [qtp-ambari-client-4139] ServiceFormattedException:99 - Can't get secure connection to https://ambari1.example.com:8443/api/v1/clusters/ClusterDemo/hosts?fields=Hosts%2Fpublic_host_name%2....  Truststore path or password is not set.
15 Jun 2016 05:30:14,862 ERROR [qtp-ambari-client-4139] ServiceFormattedException:100 - java.lang.IllegalStateException: Can't get secure connection to https://ambari1.example.com:8443/api/v1/clusters/ClusterDemo/hosts?fields=Hosts%2Fpublic_host_name%2....  Truststore path or password is not set.

java.lang.IllegalStateException: Can't get secure connection to https://ambari1.example.com:8443/api/v1/clusters/ClusterDemo/hosts?fields=Hosts%2Fpublic_host_name%2....  Truststore path or password is not set.
    at org.apache.ambari.server.controller.internal.URLStreamProvider.getSSLConnection(URLStreamProvider.java:286)
    at org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:172)
    at org.apache.ambari.server.view.ViewAmbariStreamProvider.getInputStream(ViewAmbariStreamProvider.java:118)
    at org.apache.ambari.server.view.ViewAmbariStreamProvider.readFrom(ViewAmbariStreamProvider.java:78)
    at org.apache.ambari.view.utils.ambari.URLStreamProviderBasicAuth.readFrom(URLStreamProviderBasicAuth.java:65)
    at org.apache.ambari.view.utils.ambari.AmbariApi.requestClusterAPI(AmbariApi.java:173)
    at org.apache.ambari.view.utils.ambari.AmbariApi.requestClusterAPI(AmbariApi.java:142)
    at org.apache.ambari.view.utils.ambari.AmbariApi.getHostsWithComponent(AmbariApi.java:99)
    at org.apache.ambari.view.hive.client.ConnectionFactory.getHiveHost(ConnectionFactory.java:79)
    at org.apache.ambari.view.hive.client.ConnectionFactory.create(ConnectionFactory.java:68)
    at org.apache.ambari.view.hive.client.UserLocalConnection.initialValue(UserLocalConnection.java:42)
    at org.apache.ambari.view.hive.client.UserLocalConnection.initialValue(UserLocalConnection.java:26)
    at org.apache.ambari.view.utils.UserLocal.get(UserLocal.java:66)
    at org.apache.ambari.view.hive.resources.browser.HiveBrowserService.databases(HiveBrowserService.java:87)

In order to resolve this issue lets configure the "truststore" on Ambari Server. Following are the steps to create & configure the truststore for Ambari Server:

Step-1). Use the "$JAVA_HOME/bin/keytool" to generate the truststore based on the previously created "ambari1.example.com.crt" certificate.

[root@ambari1 ambari-server]# cd /etc/ambari-server/certs/

[root@ambari1 certs]# keytool -import -file /etc/ambari-server/certs/ambari1.example.com.crt -alias ambari-server -keystore ambari-server-truststore

Enter keystore password: hadoop
Re-enter new password: hadoop
Owner: CN=ambari1.example.com, O=hwx, L=Bangalore, ST=Karnataka, C=IN
Issuer: CN=ambari1.example.com, O=hwx, L=Bangalore, ST=Karnataka, C=IN
Serial number: ffeef037fde4559f
Valid from: Wed Jun 15 00:02:13 PDT 2016 until: Thu Jun 15 00:02:13 PDT 2017
Certificate fingerprints:
     MD5:  46:44:9D:52:90:C1:D6:9D:9B:2F:11:53:7E:50:8F:16
     SHA1: 61:5E:85:B4:4D:3D:C8:7A:BB:0B:C3:48:07:D6:A4:BD:79:BD:A8:A1
     Signature algorithm name: SHA1withRSA
     Version: 1
Trust this certificate? [no]:  yes
Certificate was added to keystore

Step-2). Configuring Ambari Server to use the "/etc/ambari-server/certs/ambari-server-truststore" using the command [ambari-server setup-security] by choosing option (4) as following.

NOTE: Users should stop the ambari-server before making any truststore related configuration changes.

[root@ambari1 ~]# ambari-server stop

[root@ambari1 certs]# ambari-server setup-security
Using python  /usr/bin/python
Security setup options...
===========================================================================
Choose one of the following options:
  [1] Enable HTTPS for Ambari server.
  [2] Encrypt passwords stored in ambari.properties file.
  [3] Setup Ambari kerberos JAAS configuration.
  [4] Setup truststore.
  [5] Import certificate to truststore.
===========================================================================
Enter choice, (1-5): 4
Do you want to configure a truststore [y/n] (y)? y
TrustStore type [jks/jceks/pkcs12] (jks):jks
Path to TrustStore file :/etc/ambari-server/certs/ambari-server-truststore
Password for TrustStore: hadoop
Re-enter password: hadoop
Ambari Server 'setup-security' completed successfully.

Step-3). Start the Ambari Server.

[root@ambari1 certs]# ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start....................
Ambari Server 'start' completed successfully.

Step-4). The hive view should not be throwing the previously mentioned error now "Truststore path or password is not set" The "ambari.properties" should have the following entry in it now:

ssl.trustStore.type=jks
ssl.trustStore.path=/etc/ambari-server/certs/ambari-server-truststore

.

==== Other Issues: ====

Similarly if you notice the following kind of error:

org.apache.ambari.view.utils.ambari.AmbariApiException: RA040 I/O error while requesting Ambari
.
.
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

.

The please go through the steps described in the following article amd make sure that we imported AmbariServer certificate inside the Ambari Server's truststore.

SunCertPathBuilderException, while accessing the Ambari YARN Queue Manager view : https://community.hortonworks.com/content/supportkb/150265/errorpkix-path-building-failed-sunsecurit...

7,571 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 12:00 PM
Updated by:
 
Contributors
Top Kudoed Authors