Support Questions
Find answers, ask questions, and share your expertise

Ranger knox test connection fails when kerberos enabled on new cluster

Expert Contributor

Cluster: HDP2.5.3
I setup new cluster and enabled kerberos. Also enabled knox ranger plugin and tried test connection which fails with below error -

2018-02-02 18:55:53,821 [timed-executor-pool-0] ERROR org.apache.ranger.plugin.util.PasswordUtils (PasswordUtils.java:127) - Unable to decrypt password due to error
javax.crypto.IllegalBlockSizeException: Input length must be multiple of 8 when decrypting with padded cipher
        at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:936)
        at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:847)
        at com.sun.crypto.provider.PBES1Core.doFinal(PBES1Core.java:416)
        at com.sun.crypto.provider.PBEWithMD5AndDESCipher.engineDoFinal(PBEWithMD5AndDESCipher.java:316)
        at javax.crypto.Cipher.doFinal(Cipher.java:2165)
        at org.apache.ranger.plugin.util.PasswordUtils.decryptPassword(PasswordUtils.java:112)
        at org.apache.ranger.services.knox.client.KnoxClient.getTopologyList(KnoxClient.java:79)
        at org.apache.ranger.services.knox.client.KnoxClient$2.call(KnoxClient.java:397)
        at org.apache.ranger.services.knox.client.KnoxClient$2.call(KnoxClient.java:394)
        at org.apache.ranger.services.knox.client.KnoxClient.timedTask(KnoxClient.java:423)
        at org.apache.ranger.services.knox.client.KnoxClient.getKnoxResources(KnoxClient.java:402)
        at org.apache.ranger.services.knox.client.KnoxClient.connectionTest(KnoxClient.java:311)
        at org.apache.ranger.services.knox.client.KnoxResourceMgr.validateConfig(KnoxResourceMgr.java:43)
        at org.apache.ranger.services.knox.RangerServiceKnox.validateConfig(RangerServiceKnox.java:56)
        at org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:560)
        at org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:547)
        at org.apache.ranger.biz.ServiceMgr$TimedCallable.call(ServiceMgr.java:508)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
2018-02-02 18:55:53,822 [timed-executor-pool-0] INFO  apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:81) - Password decryption failed; trying knox connection with received password string
2018-02-02 18:55:53,906 [timed-executor-pool-0] ERROR apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:158) - Exception on REST call to KnoxUrl : https://ip-10-0-1-157.ec2.internal:8443/gateway/admin/api/v1/topologies.
com.sun.jersey.api.client.ClientHandlerException: javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
        at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:131)
        at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:81)
        at com.sun.jersey.api.client.Client.handle(Client.java:616)
        at com.sun.jersey.api.client.WebResource.handle(WebResource.java:559)
        at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:72)
        at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:454)
        at org.apache.ranger.services.knox.client.KnoxClient.getTopologyList(KnoxClient.java:98)
        at org.apache.ranger.services.knox.client.KnoxClient$2.call(KnoxClient.java:397)
        at org.apache.ranger.services.knox.client.KnoxClient$2.call(KnoxClient.java:394)
        at org.apache.ranger.services.knox.client.KnoxClient.timedTask(KnoxClient.java:423)
        at org.apache.ranger.services.knox.client.KnoxClient.getKnoxResources(KnoxClient.java:402)
        at org.apache.ranger.services.knox.client.KnoxClient.connectionTest(KnoxClient.java:311)
        at org.apache.ranger.services.knox.client.KnoxResourceMgr.validateConfig(KnoxResourceMgr.java:43)
        at org.apache.ranger.services.knox.RangerServiceKnox.validateConfig(RangerServiceKnox.java:56)
        at org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:560)
        at org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:547)
        at org.apache.ranger.biz.ServiceMgr$TimedCallable.call(ServiceMgr.java:508)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1959)
        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1916)
        at sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1899)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1420)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
        at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1564)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
        at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:347)
        at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:218)
        at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:129)
        ... 20 more
Caused by: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
        at sun.security.validator.PKIXValidator.<init>(PKIXValidator.java:91)
        at sun.security.validator.Validator.getInstance(Validator.java:179)
        at sun.security.ssl.X509TrustManagerImpl.getValidator(X509TrustManagerImpl.java:312)
        at sun.security.ssl.X509TrustManagerImpl.checkTrustedInit(X509TrustManagerImpl.java:171)
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:184)
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1496)
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026)
        at sun.security.ssl.Handshaker.process_record(Handshaker.java:961)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1072)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
        ... 29 more
Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
        at java.security.cert.PKIXParameters.setTrustAnchors(PKIXParameters.java:200)
        at java.security.cert.PKIXParameters.<init>(PKIXParameters.java:120)
        at java.security.cert.PKIXBuilderParameters.<init>(PKIXBuilderParameters.java:104)
        at sun.security.validator.PKIXValidator.<init>(PKIXValidator.java:89)
        ... 41 more
2018-02-02 18:55:53,907 [timed-executor-pool-0] ERROR apache.ranger.services.knox.client.KnoxResourceMgr (KnoxResourceMgr.java:45) - <== KnoxResourceMgr.connectionTest Error: org.apache.ranger.plugin.client.HadoopException: Exception on REST call to KnoxUrl : https://ip-10-0-1-157.ec2.internal:8443/gateway/admin/api/v1/topologies.
2018-02-02 18:55:53,907 [timed-executor-pool-0] ERROR org.apache.ranger.services.knox.RangerServiceKnox (RangerServiceKnox.java:58) - <== RangerServiceKnox.validateConfig Error:org.apache.ranger.plugin.client.HadoopException: Exception on REST call to KnoxUrl : https://ip-10-0-1-157.ec2.internal:8443/gateway/admin/api/v1/topologies.
2018-02-02 18:55:53,907 [timed-executor-pool-0] ERROR org.apache.ranger.biz.ServiceMgr$TimedCallable (ServiceMgr.java:510) - TimedCallable.call: Error:org.apache.ranger.plugin.client.HadoopException: Exception on REST call to KnoxUrl : https://ip-10-0-1-157.ec2.internal:8443/gateway/admin/api/v1/topologies.
2018-02-02 18:55:53,908 [http-bio-6080-exec-10] ERROR org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:188) - ==> ServiceMgr.validateConfig Error:org.apache.ranger.plugin.client.HadoopException: org.apache.ranger.plugin.client.HadoopException: Exception on REST call to KnoxUrl : https://ip-10-0-1-157.ec2.internal:8443/gateway/admin/api/v1/topologies.


Is this default behaviour ?

5 REPLIES 5

Re: Ranger knox test connection fails when kerberos enabled on new cluster

Super Collaborator

Ranger admin has to connect on https url "https://ip-10-0-1-157.ec2.internal:8443/gateway/admin/api/v1/topologies."

Import knox host's certificate to ranger.truststore.file

On Ranger admin host execute below command and restart ranger admin service to reread the truststore.

#echo | openssl s_client -connect ip-10-0-1-157.ec2.internal:8443 2>&1 | sed --quiet '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > knox.crt
#keytool -import -file knox.crt -keystore <keystore set with ranger.truststore.file> -alias knox

Re: Ranger knox test connection fails when kerberos enabled on new cluster

Expert Contributor
@rguruvannagari

Hi, What should be default value of "<keystore set with ranger.truststore.file>" ?

since i haven't set any keystore/trustore for ranger.

Re: Ranger knox test connection fails when kerberos enabled on new cluster

Cloudera Employee

In Ambari, the default value is

/etc/ranger/admin/conf/ranger-admin-keystore.jks

77398-y6uvz.png

Re: Ranger knox test connection fails when kerberos enabled on new cluster

Sagar Shimpi you should just setup a new trust store even if you env is not SSL enabled , and add the knox gateway certificate into the ranger trust store and configure that truststore file into the ranger.truststore.file property in ranger admin conf. BTW you will have to create a fresh ranger trust store file I guess because your env is non ssl and it will not be present already.

Re: Ranger knox test connection fails when kerberos enabled on new cluster

Expert Contributor

@Deepak Sharma Thanks. Will try and revert.