I have check the Ambari Server log but it's not really very helpful. sudo $JAVA_HOME/bin/keytool -list -v -keystore /var/lib/ambari-server/keys/ambari-server-truststore > /tmp/05122018_Ambari_truststore_cert Confirmed the truststore location matches the ambari.properties location under /etc/ambari-server/conf/ambari.properties. @Robert Levas , Your suggestion might help but do you not recon it might cause issues later down the line? Feels like it would be a bit "hacky" 🙂 .. Kind Regards 05 Dec 2018 14:56:00,217 ERROR [ambari-client-thread-303] KerberosHelperImpl:2232 - Cannot validate credentials: org.apache.ambari.server.serveraction.kerberos.KerberosInvalidConfigurationException: Failed to connect to KDC - Failed to communicate with the Active Directory at ldaps://ad-serverxxxx:636: simple bind failed ad-serverxxxx:636 Make sure the server's SSL certificate or CA certificates have been imported into Ambari's truststore. 05 Dec 2018 14:56:00,217 ERROR [ambari-client-thread-303] BaseManagementHandler:67 - Bad request received: Failed to connect to KDC - Failed to communicate with the Active Directory at ldaps://ad-serverxxxx:636 simple bind failed: ad-serverxxx:636 Make sure the server's SSL certificate or CA certificates have been imported into Ambari's truststore. 05 Dec 2018 15:02:51,205 INFO [ambari-client-thread-554] AmbariManagementControllerImpl:4173 - Received action execution request, clusterName=caphdpoc, request=isCommand :true, action :null, command :KERBEROS_SERVICE_CHECK, inputs :{HAS_RESOURCE_FILTERS=true}, resourceFilters: [RequestResourceFilter{serviceName='KERBEROS', componentName='null', hostNames=[]}], exclusive: false, clusterName :caphdppoc 05 Dec 2018 15:02:51,364 WARN [ambari-client-thread-554] ADKerberosOperationHandler:470 - Failed to communicate with the Active Directory at ldaps://ad-serverxxxx:636:: simple bind failed: ad-serverxxxx:636 javax.naming.CommunicationException: simple bind failed: ad-serverxxxx:636 [Root exception is javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target] at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:219) at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2791) at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:319) at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:192) at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:210) at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:153) at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:83) at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684) at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:313) at javax.naming.InitialContext.init(InitialContext.java:244) at javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:154) at org.apache.ambari.server.serveraction.kerberos.ADKerberosOperationHandler.createInitialLdapContext(ADKerberosOperationHandler.java:514) at org.apache.ambari.server.serveraction.kerberos.ADKerberosOperationHandler.createLdapContext(ADKerberosOperationHandler.java:465) at org.apache.ambari.server.serveraction.kerberos.ADKerberosOperationHandler.open(ADKerberosOperationHandler.java:182) at ...... com.sun.jndi.ldap.Connection.writeRequest(Connection.java:416) at com.sun.jndi.ldap.LdapClient.ldapBind(LdapClient.java:359) at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:214) ... 114 more Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397) at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302) at sun.security.validator.Validator.validate(Validator.java:262) at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324) at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229) at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621) ... 127 more Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280) at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392) ... 133 more 05 Dec 2018 15:02:51,367 ERROR [ambari-client-thread-554] KerberosHelperImpl:2232 - Cannot validate credentials: org.apache.ambari.server.serveraction.kerberos.KerberosInvalidConfigurationException: Failed to connect to KDC - Failed to communicate with the Active Directory at ldaps://ad-serverxxxx:636 simple bind failed: ad-serverxxxx:636 Make sure the server's SSL certificate or CA certificates have been imported into Ambari's truststore. 05 Dec 2018 15:02:51,367 ERROR [ambari-client-thread-554] BaseManagementHandler:67 - Bad request received: Failed to connect to KDC - Failed to communicate with the Active Directory at ldap://ad-serverxxxx:636: simple bind failed: ad-serverxxxx:636 Make sure the server's SSL certificate or CA certificates have been imported into Ambari's truststore. ... View more

Hi Mark, dealing with Spark Thrift issue too on v2.5.3. I noticed my times are not in sync on all the nodes. I'm behind a proxy so my ntpd wasn't updating. Probably not your issue but might be good to keep in mind. 17/02/23 12:24:07 INFO Client: client token: N/A diagnostics: Application application_1487754339887_0003 failed 2 times due to Error launching appattempt_1487754339887_0003_000002. Got exception: org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container. This token is expired. current time is 1487845447016 found 1487756740753 Note: System times on machines may be out of sync. Check system time and time zones. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:168) at org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106) at org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.launch(AMLauncher.java:122) at org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.run(AMLauncher.java:250) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) . Failing the application. ApplicationMaster host: N/A ApplicationMaster RPC port: -1 queue: default start time: 1487756139363 final status: FAILED tracking URL: http://HNode2.quasar.com:8088/cluster/app/application_1487754339887_0003 user: hive ... View more