Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Solved Go to solution

Ranger Knox Plugin is failing in test connection while creating service -> Unable to retrieve any topologies/services using given parameters.

Labels:
avatar
author-rank nhgodwal
Contributor

Created ‎02-26-2018 12:47 AM

This is the knoxurl I am giving while creating service in knox plugin: https://localhost:8443/gateway/admin/api/v1/topologies

I have also added group info in topologies/admin.xml as per suggestion here (https://community.hortonworks.com/articles/38348/ranger-is-not-allowing-access-to-knox-resources-wh.html)

	    <param>
                <name>main.ldapRealm.authorizationEnabled</name>
                <value>true</value>
            </param>
            <param>
                <name>main.ldapRealm.groupSearchBase</name>
                <value>ou=groups,dc=hadoop,dc=apache,dc=org</value>
            </param>
            <param>
                <name>main.ldapRealm.groupObjectClass</name>
                <value>group</value>
            </param>
            <param>
                <name>main.ldapRealm.groupIdAttribute</name>
                <value>cn</value>
            </param

Below are the log details:

ranger-admin.log

2018-02-26 00:34:47,535 [timed-executor-pool-0] INFO  apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:81)
 - Password decryption failed; trying knox connection with received password string
2018-02-26 00:34:47,632 [timed-executor-pool-0] ERROR apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:131) - Got invalid REST response from: https://localhost:8443/gateway/admin/api/v1/topologies, responseStatus: 403

gateway.log

2018-02-26 00:34:47,614 INFO  hadoop.gateway (KnoxLdapRealm.java:getUserDn(691)) - Computed userDn: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: admin
2018-02-26 00:34:47,630 ERROR hadoop.gateway (KnoxLdapRealm.java:getRoles(246)) - Failed to get system ldap connection: javax.naming.AuthenticationException: [LDAP: error code 49 - INVALID_CREDENTIALS: Bind failed: ERR_229 Cannot authenticate user ]

gateway-audit.log

18/02/26 00:24:44 ||65cc6da4-9fa6-4e6d-8b69-b99f5d9acacb|audit|127.0.0.1|KNOX|admin|||authentication|uri|/gateway/admin/api/v1/topologies|success|Groups: [] 18/02/26 00:24:44 ||65cc6da4-9fa6-4e6d-8b69-b99f5d9acacb|audit|127.0.0.1|KNOX|admin|||access|uri|/gateway/admin/api/v1/topologies|success|Response status: 403
10,017 Views
0 Kudos
1 ACCEPTED SOLUTION

avatar
author-rank VR46 author-rank
Guru

Created ‎03-15-2018 02:28 AM

Hello @GN_Exp,

There are couple of things here:

1. From your gateway.log (dt:03/12), it looks like the Knox Gateway is trying to initialize RangerPDPKnoxFilter in the Gateway request filter and failing while doing so. This RangerPDPKnoxFilter is used when Kerberos is configured. Since you don't have Kerberos configured, you should not be using this.

2. To enable the Ranger plugin in Knox gateway service, you do not always need XAsecurePDPKnox as authorization provider. "AclsAuthz" would do just fine. This is usually used for Service Level Authorization in Knox topology, which you don't need for Ranger plugin test connection.

Therefore, please stick to "AclsAuthz" unless you have any other use-case. If you still have problem with Knox service repo in Ranger, please attach the screenshot of Knox repo configuration from Ranger UI and screenshot of the error (if any).

Hope this helps !

View solution in original post

10,404 Views
0 Kudos
14 REPLIES 14

avatar
author-rank dsharma author-rank
Guru

Created ‎02-27-2018 08:25 PM

@GN_Exp is this cluster an unsecure cluster ? if so the can you please validate the username and password given in the knox repo ?

8,101 Views

avatar
author-rank nhgodwal
Contributor

Created ‎03-07-2018 07:10 PM

@Deepak Sharma

Yes, the cluster is not Kerberos secured. I have validated the username and corrected the password I was giving while creating service. Now, I am not able to see any error in gateway.log. But gateway-audit.log and ranger-admin.log still showing same error.

Can you suggest?

8,101 Views
0 Kudos

avatar
author-rank nhgodwal
Contributor

Created ‎03-08-2018 11:52 PM

@Deepak Sharma

i am using same username and password as I am using for HDFS service.

8,101 Views
0 Kudos

avatar
author-rank dsharma author-rank
Guru

Created ‎03-11-2018 06:57 PM

GN_Exp is there ranger knox policy present to allow access to admin user the access of ?

8,101 Views
0 Kudos

avatar
author-rank nhgodwal
Contributor

Created ‎03-11-2018 10:51 PM

@Deepak Sharma Thanks for replying.

Yes, when I create knox service, it automatically creates a policy to allow access to admin. But it didn't show in Audit tab. I am attaching screen shots too.

Also, this time I did the fresh intsallation of ranger, hdfs-plugin and then knox plugin. I find logs little different then previous.

gateway-audit.log

18/03/11 22:57:46 ||bf9148a8-fe74-4a1a-b18f-9a300f0a062c|audit|127.0.0.1|KNOX||||access|uri|/gateway/admin/api/v1/topologies|unavailable|Request method: GET
18/03/11 22:57:46 ||bf9148a8-fe74-4a1a-b18f-9a300f0a062c|audit|127.0.0.1|KNOX||||authentication|principal|admin|failure|LDAP authentication failed.
18/03/11 22:57:46 ||bf9148a8-fe74-4a1a-b18f-9a300f0a062c|audit|127.0.0.1|KNOX||||access|uri|/gateway/admin/api/v1/topologies|success|Response status: 401

gateway.log

2018-03-11 22:57:46,092 INFO  hadoop.gateway (KnoxLdapRealm.java:getUserDn(691)) - Computed userDn: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: admin
2018-03-11 22:57:46,118 INFO  hadoop.gateway (KnoxLdapRealm.java:doGetAuthenticationInfo(203)) - Could not login: org.apache.shiro.authc.UsernamePasswordToken - admin, rememberMe=false (127.0.0.1)
2018-03-11 22:57:46,121 ERROR hadoop.gateway (KnoxLdapRealm.java:doGetAuthenticationInfo(205)) - Shiro unable to login: javax.naming.AuthenticationException: [LDAP: error code 49 - INVALID_CREDENTIALS: Bind failed: ERR_229 Cannot authenticate user uid=admin,ou=people,dc=hadoop,dc=apache,dc=org]


ranger_admin.log

2018-03-11 22:57:45,291 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/ranger-knox-plugin-0.7.0.2.6.4.0-91.jar
2018-03-11 22:57:45,291 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/jackson-core-asl-1.9.13.jar
2018-03-11 22:57:45,292 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/commons-collections-3.2.2.jar
2018-03-11 22:57:45,293 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/jackson-mapper-asl-1.9.13.jar
2018-03-11 22:57:45,293 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/commons-lang-2.6.jar
2018-03-11 22:57:46,142 [timed-executor-pool-0] ERROR apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:131) - Got invalid REST response from: https://localhost:8443/gateway/admin/api/v1/topologies, responseStatus: 401


I installed the ranger and plugins referring this link -> https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_command-line-installation/content/ch_ins...

Please let me know if you need me to post any config files.


audit-tab.pngpolicy-knox.png
8,101 Views
0 Kudos

avatar
author-rank dsharma author-rank
Guru

Created ‎03-12-2018 06:05 AM

I think you are using LDAP that comes with knox , can you please check if it is up and running ? and if yes then check for that admin user password in users.ldif file in /etc/knox/conf. by defaut it is admin-password

8,101 Views
0 Kudos

avatar
author-rank nhgodwal
Contributor

Created ‎03-12-2018 02:45 PM

@Deepak Sharma @vsuvagia

yes, I am using demo LDAP comes with knox.

I changed the password from admin to admin-password. Now I am getting different error logs:

Even changing the localhost to fqdn giving me same error logs.

ranger-admin.log

2018-03-12 14:34:55,969 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/ranger-knox-plugin-0.7.0.2.6.4.0-91.jar
2018-03-12 14:34:55,970 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/jackson-core-asl-1.9.13.jar
2018-03-12 14:34:55,970 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/commons-collections-3.2.2.jar
2018-03-12 14:34:55,970 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/jackson-mapper-asl-1.9.13.jar
2018-03-12 14:34:55,970 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/commons-lang-2.6.jar
2018-03-12 14:34:56,013 [timed-executor-pool-0] ERROR org.apache.ranger.plugin.util.PasswordUtils (PasswordUtils.java:130) - Unable to decrypt password due to error
javax.crypto.IllegalBlockSizeException: Input length must be multiple of 8 when decrypting with padded cipher
        at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:936)
        at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:847)
        at com.sun.crypto.provider.PBES1Core.doFinal(PBES1Core.java:416)
        at com.sun.crypto.provider.PBEWithMD5AndDESCipher.engineDoFinal(PBEWithMD5AndDESCipher.java:316)
        at javax.crypto.Cipher.doFinal(Cipher.java:2165)
        at org.apache.ranger.plugin.util.PasswordUtils.decryptPassword(PasswordUtils.java:115)
        at org.apache.ranger.services.knox.client.KnoxClient.getTopologyList(KnoxClient.java:79)
        at org.apache.ranger.services.knox.client.KnoxClient$2.call(KnoxClient.java:406)
        at org.apache.ranger.services.knox.client.KnoxClient$2.call(KnoxClient.java:402)
        at org.apache.ranger.services.knox.client.KnoxClient.timedTask(KnoxClient.java:431)
        at org.apache.ranger.services.knox.client.KnoxClient.getKnoxResources(KnoxClient.java:410)
        at org.apache.ranger.services.knox.client.KnoxClient.connectionTest(KnoxClient.java:315)
        at org.apache.ranger.services.knox.client.KnoxResourceMgr.validateConfig(KnoxResourceMgr.java:43)
        at org.apache.ranger.services.knox.RangerServiceKnox.validateConfig(RangerServiceKnox.java:56)
        at org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:560)
        at org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:547)
        at org.apache.ranger.biz.ServiceMgr$TimedCallable.call(ServiceMgr.java:508)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
2018-03-12 14:34:56,015 [timed-executor-pool-0] INFO  apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:81) - Password decryption failed; trying knox connection with received password string
2018-03-12 14:34:57,918 [timed-executor-pool-0] ERROR apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:131) - Got invalid REST response from: https://localhost:8443/gateway/admin/api/v1/topologies, responseStatus: 500

gateway-audit.log

18/03/12 14:34:56 ||3e3df102-de59-4c19-9779-cdd4522181bb|audit|127.0.0.1|KNOX||||access|uri|/gateway/admin/api/v1/topologies|unavailable|Request method: GET
18/03/12 14:34:56 ||3e3df102-de59-4c19-9779-cdd4522181bb|audit|127.0.0.1|KNOX|admin|||authentication|uri|/gateway/admin/api/v1/topologies|success|
18/03/12 14:34:56 ||3e3df102-de59-4c19-9779-cdd4522181bb|audit|127.0.0.1|KNOX|admin|||authentication|uri|/gateway/admin/api/v1/topologies|success|Groups: []
18/03/12 14:34:57 ||3e3df102-de59-4c19-9779-cdd4522181bb|audit|127.0.0.1|KNOX|admin|||access|uri|/gateway/admin/api/v1/topologies|failure|

gateway.log

gateway-log.txt

Preview file
34 KB
8,101 Views
0 Kudos

avatar
author-rank vsuvagia author-rank
Rising Star

Created ‎03-13-2018 10:07 AM

@GN_Exp, has the Knox certificate been imported in Ranger truststore ?
If not here are some links you can follow:

installing ranger-knox plugin.

ranger-knox-repo-test-connection-failure.

do-we-need-to-add-knox-self-signed-cert-into-range.

8,101 Views
0 Kudos

avatar
author-rank nhgodwal
Contributor

Created ‎03-13-2018 02:46 PM

@vsuvagia

Yes, I did import the knox cert to Ranger truststore. I am not getting any SSL error.

8,101 Views
0 Kudos
Announcements
What's New @ Cloudera
Cloudera Data Engineering 1.23: Access Spark from Your Favor...
What's New @ Cloudera
HBase REST server scaling support is Generally Available
What's New @ Cloudera
New CLI option in the update-database command
What's New @ Cloudera
New Action menu item in the Cloudera Operational Database UI
What's New @ Cloudera
Get the list of supported instance types in the Cloudera Ope...
View More Announcements