certificate_unknown error observed on enabling TLS/SSL on hadoop and yarn

[#1]

I am using Cloudera 5.7 and have installed the basic services successfully. But when i enable TLS/SSL for hadoop and yarn, I start getting below exception from NameNode and Resource Manager.

WARN org.mortbay.log: EXCEPTION
javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
        at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:1959)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1077)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
        at org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:723)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

I am using self-signed certificates which I have generated using instructions in Example3: http://www.cloudera.com/documentation/enterprise/latest/topics/cm_sg_create_key_trust.html. Both keystore and truststore are located in /var/lib/hadoop-hdfs/certs on each node of cluster and have all read permissions and hadoop as owner group.

# ls -l node.keystore
-rwxrwxrwx 1 hdfs hadoop 1317 Apr 11 06:32 node.keystore

Please help.

[#2]

Hello @vidhatajain,

Please check that your self-signed CA certificate has been added to your java truststore on all hosts.

Example:

keytool -v -list -keystore /usr/java/jdk1.7.0_67-cloudera/jre/lib/security/jssecacerts

Gabor

[#3]

The certificate_unknown message is received as an alert from the caller initiating the TLS session.  Generally, that means that the client making a connection to the server did not trust the certificate.

To find out who is really not trusting the NameNode certificate, check anything that connects to the NameNode.  Mostly, it is the DataNodes who need to hearbeat in I think.  Check your DataNode logs to find out if you get exceptions regarding trust when they attempt to make a connection.  For Resource Manager, check the NodeMangers' logs too.

Once you have reviewed the logs, you will likely have a better idea of what is going on.

You mentioned that "/var/lib/hadoop-hdfs/certs" is your truststore.  If it is, it should contain the certificate for every host in your cluster.  Also, make sure you have configured a path to it in the Service Configuration in HDFS:

"Cluster-Wide Default TLS/SSL Client Truststore Location"