Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Ranger Knox Plugin is failing in test connection while creating service -> Unable to retrieve any topologies/services using given parameters.

avatar
Contributor

This is the knoxurl I am giving while creating service in knox plugin: https://localhost:8443/gateway/admin/api/v1/topologies

I have also added group info in topologies/admin.xml as per suggestion here (https://community.hortonworks.com/articles/38348/ranger-is-not-allowing-access-to-knox-resources-wh.html)

	    <param>
                <name>main.ldapRealm.authorizationEnabled</name>
                <value>true</value>
            </param>
            <param>
                <name>main.ldapRealm.groupSearchBase</name>
                <value>ou=groups,dc=hadoop,dc=apache,dc=org</value>
            </param>
            <param>
                <name>main.ldapRealm.groupObjectClass</name>
                <value>group</value>
            </param>
            <param>
                <name>main.ldapRealm.groupIdAttribute</name>
                <value>cn</value>
            </param

Below are the log details:

ranger-admin.log

2018-02-26 00:34:47,535 [timed-executor-pool-0] INFO  apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:81)
 - Password decryption failed; trying knox connection with received password string
2018-02-26 00:34:47,632 [timed-executor-pool-0] ERROR apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:131) - Got invalid REST response from: https://localhost:8443/gateway/admin/api/v1/topologies, responseStatus: 403

gateway.log

2018-02-26 00:34:47,614 INFO  hadoop.gateway (KnoxLdapRealm.java:getUserDn(691)) - Computed userDn: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: admin
2018-02-26 00:34:47,630 ERROR hadoop.gateway (KnoxLdapRealm.java:getRoles(246)) - Failed to get system ldap connection: javax.naming.AuthenticationException: [LDAP: error code 49 - INVALID_CREDENTIALS: Bind failed: ERR_229 Cannot authenticate user ]

gateway-audit.log

18/02/26 00:24:44 ||65cc6da4-9fa6-4e6d-8b69-b99f5d9acacb|audit|127.0.0.1|KNOX|admin|||authentication|uri|/gateway/admin/api/v1/topologies|success|Groups: [] 18/02/26 00:24:44 ||65cc6da4-9fa6-4e6d-8b69-b99f5d9acacb|audit|127.0.0.1|KNOX|admin|||access|uri|/gateway/admin/api/v1/topologies|success|Response status: 403
1 ACCEPTED SOLUTION

avatar
Guru

Hello @GN_Exp,

There are couple of things here:

1. From your gateway.log (dt:03/12), it looks like the Knox Gateway is trying to initialize RangerPDPKnoxFilter in the Gateway request filter and failing while doing so. This RangerPDPKnoxFilter is used when Kerberos is configured. Since you don't have Kerberos configured, you should not be using this.

2. To enable the Ranger plugin in Knox gateway service, you do not always need XAsecurePDPKnox as authorization provider. "AclsAuthz" would do just fine. This is usually used for Service Level Authorization in Knox topology, which you don't need for Ranger plugin test connection.

Therefore, please stick to "AclsAuthz" unless you have any other use-case. If you still have problem with Knox service repo in Ranger, please attach the screenshot of Knox repo configuration from Ranger UI and screenshot of the error (if any).

Hope this helps !

View solution in original post

14 REPLIES 14

avatar

@GN_Exp is this cluster an unsecure cluster ? if so the can you please validate the username and password given in the knox repo ?

avatar
Contributor
@Deepak Sharma

Yes, the cluster is not Kerberos secured. I have validated the username and corrected the password I was giving while creating service. Now, I am not able to see any error in gateway.log. But gateway-audit.log and ranger-admin.log still showing same error.

Can you suggest?

avatar
Contributor

@Deepak Sharma

i am using same username and password as I am using for HDFS service.

avatar

GN_Exp is there ranger knox policy present to allow access to admin user the access of ?

avatar
Contributor

@Deepak Sharma Thanks for replying.

Yes, when I create knox service, it automatically creates a policy to allow access to admin. But it didn't show in Audit tab. I am attaching screen shots too.

Also, this time I did the fresh intsallation of ranger, hdfs-plugin and then knox plugin. I find logs little different then previous.

gateway-audit.log

18/03/11 22:57:46 ||bf9148a8-fe74-4a1a-b18f-9a300f0a062c|audit|127.0.0.1|KNOX||||access|uri|/gateway/admin/api/v1/topologies|unavailable|Request method: GET
18/03/11 22:57:46 ||bf9148a8-fe74-4a1a-b18f-9a300f0a062c|audit|127.0.0.1|KNOX||||authentication|principal|admin|failure|LDAP authentication failed.
18/03/11 22:57:46 ||bf9148a8-fe74-4a1a-b18f-9a300f0a062c|audit|127.0.0.1|KNOX||||access|uri|/gateway/admin/api/v1/topologies|success|Response status: 401

gateway.log

2018-03-11 22:57:46,092 INFO  hadoop.gateway (KnoxLdapRealm.java:getUserDn(691)) - Computed userDn: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: admin
2018-03-11 22:57:46,118 INFO hadoop.gateway (KnoxLdapRealm.java:doGetAuthenticationInfo(203)) - Could not login: org.apache.shiro.authc.UsernamePasswordToken - admin, rememberMe=false (127.0.0.1)
2018-03-11 22:57:46,121 ERROR hadoop.gateway (KnoxLdapRealm.java:doGetAuthenticationInfo(205)) - Shiro unable to login: javax.naming.AuthenticationException: [LDAP: error code 49 - INVALID_CREDENTIALS: Bind failed: ERR_229 Cannot authenticate user uid=admin,ou=people,dc=hadoop,dc=apache,dc=org]


ranger_admin.log

2018-03-11 22:57:45,291 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/ranger-knox-plugin-0.7.0.2.6.4.0-91.jar
2018-03-11 22:57:45,291 [http-bio-6080-exec-9] WARN org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/jackson-core-asl-1.9.13.jar
2018-03-11 22:57:45,292 [http-bio-6080-exec-9] WARN org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/commons-collections-3.2.2.jar
2018-03-11 22:57:45,293 [http-bio-6080-exec-9] WARN org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/jackson-mapper-asl-1.9.13.jar
2018-03-11 22:57:45,293 [http-bio-6080-exec-9] WARN org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/commons-lang-2.6.jar
2018-03-11 22:57:46,142 [timed-executor-pool-0] ERROR apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:131) - Got invalid REST response from: https://localhost:8443/gateway/admin/api/v1/topologies, responseStatus: 401


I installed the ranger and plugins referring this link -> https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_command-line-installation/content/ch_ins...

Please let me know if you need me to post any config files.


audit-tab.pngpolicy-knox.png

avatar

I think you are using LDAP that comes with knox , can you please check if it is up and running ? and if yes then check for that admin user password in users.ldif file in /etc/knox/conf. by defaut it is admin-password

avatar
Contributor

@Deepak Sharma @vsuvagia

yes, I am using demo LDAP comes with knox.

I changed the password from admin to admin-password. Now I am getting different error logs:

Even changing the localhost to fqdn giving me same error logs.

ranger-admin.log

2018-03-12 14:34:55,969 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/ranger-knox-plugin-0.7.0.2.6.4.0-91.jar
2018-03-12 14:34:55,970 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/jackson-core-asl-1.9.13.jar
2018-03-12 14:34:55,970 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/commons-collections-3.2.2.jar
2018-03-12 14:34:55,970 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/jackson-mapper-asl-1.9.13.jar
2018-03-12 14:34:55,970 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/commons-lang-2.6.jar
2018-03-12 14:34:56,013 [timed-executor-pool-0] ERROR org.apache.ranger.plugin.util.PasswordUtils (PasswordUtils.java:130) - Unable to decrypt password due to error
javax.crypto.IllegalBlockSizeException: Input length must be multiple of 8 when decrypting with padded cipher
        at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:936)
        at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:847)
        at com.sun.crypto.provider.PBES1Core.doFinal(PBES1Core.java:416)
        at com.sun.crypto.provider.PBEWithMD5AndDESCipher.engineDoFinal(PBEWithMD5AndDESCipher.java:316)
        at javax.crypto.Cipher.doFinal(Cipher.java:2165)
        at org.apache.ranger.plugin.util.PasswordUtils.decryptPassword(PasswordUtils.java:115)
        at org.apache.ranger.services.knox.client.KnoxClient.getTopologyList(KnoxClient.java:79)
        at org.apache.ranger.services.knox.client.KnoxClient$2.call(KnoxClient.java:406)
        at org.apache.ranger.services.knox.client.KnoxClient$2.call(KnoxClient.java:402)
        at org.apache.ranger.services.knox.client.KnoxClient.timedTask(KnoxClient.java:431)
        at org.apache.ranger.services.knox.client.KnoxClient.getKnoxResources(KnoxClient.java:410)
        at org.apache.ranger.services.knox.client.KnoxClient.connectionTest(KnoxClient.java:315)
        at org.apache.ranger.services.knox.client.KnoxResourceMgr.validateConfig(KnoxResourceMgr.java:43)
        at org.apache.ranger.services.knox.RangerServiceKnox.validateConfig(RangerServiceKnox.java:56)
        at org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:560)
        at org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:547)
        at org.apache.ranger.biz.ServiceMgr$TimedCallable.call(ServiceMgr.java:508)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
2018-03-12 14:34:56,015 [timed-executor-pool-0] INFO  apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:81) - Password decryption failed; trying knox connection with received password string
2018-03-12 14:34:57,918 [timed-executor-pool-0] ERROR apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:131) - Got invalid REST response from: https://localhost:8443/gateway/admin/api/v1/topologies, responseStatus: 500

gateway-audit.log

18/03/12 14:34:56 ||3e3df102-de59-4c19-9779-cdd4522181bb|audit|127.0.0.1|KNOX||||access|uri|/gateway/admin/api/v1/topologies|unavailable|Request method: GET
18/03/12 14:34:56 ||3e3df102-de59-4c19-9779-cdd4522181bb|audit|127.0.0.1|KNOX|admin|||authentication|uri|/gateway/admin/api/v1/topologies|success|
18/03/12 14:34:56 ||3e3df102-de59-4c19-9779-cdd4522181bb|audit|127.0.0.1|KNOX|admin|||authentication|uri|/gateway/admin/api/v1/topologies|success|Groups: []
18/03/12 14:34:57 ||3e3df102-de59-4c19-9779-cdd4522181bb|audit|127.0.0.1|KNOX|admin|||access|uri|/gateway/admin/api/v1/topologies|failure|

gateway.log

gateway-log.txt

avatar
Rising Star

@GN_Exp, has the Knox certificate been imported in Ranger truststore ?
If not here are some links you can follow:

installing ranger-knox plugin.

ranger-knox-repo-test-connection-failure.

do-we-need-to-add-knox-self-signed-cert-into-range.

avatar
Contributor

@vsuvagia

Yes, I did import the knox cert to Ranger truststore. I am not getting any SSL error.