Support Questions

nhgodwal · ‎02-26-2018

This is the knoxurl I am giving while creating service in knox plugin: https://localhost:8443/gateway/admin/api/v1/topologies

I have also added group info in topologies/admin.xml as per suggestion here (https://community.hortonworks.com/articles/38348/ranger-is-not-allowing-access-to-knox-resources-wh.html)

	    <param>
                <name>main.ldapRealm.authorizationEnabled</name>
                <value>true</value>
            </param>
            <param>
                <name>main.ldapRealm.groupSearchBase</name>
                <value>ou=groups,dc=hadoop,dc=apache,dc=org</value>
            </param>
            <param>
                <name>main.ldapRealm.groupObjectClass</name>
                <value>group</value>
            </param>
            <param>
                <name>main.ldapRealm.groupIdAttribute</name>
                <value>cn</value>
            </param

Below are the log details:

ranger-admin.log

2018-02-26 00:34:47,535 [timed-executor-pool-0] INFO  apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:81)
 - Password decryption failed; trying knox connection with received password string
2018-02-26 00:34:47,632 [timed-executor-pool-0] ERROR apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:131) - Got invalid REST response from: https://localhost:8443/gateway/admin/api/v1/topologies, responseStatus: 403

gateway.log

2018-02-26 00:34:47,614 INFO  hadoop.gateway (KnoxLdapRealm.java:getUserDn(691)) - Computed userDn: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: admin
2018-02-26 00:34:47,630 ERROR hadoop.gateway (KnoxLdapRealm.java:getRoles(246)) - Failed to get system ldap connection: javax.naming.AuthenticationException: [LDAP: error code 49 - INVALID_CREDENTIALS: Bind failed: ERR_229 Cannot authenticate user ]

gateway-audit.log

18/02/26 00:24:44 ||65cc6da4-9fa6-4e6d-8b69-b99f5d9acacb|audit|127.0.0.1|KNOX|admin|||authentication|uri|/gateway/admin/api/v1/topologies|success|Groups: [] 18/02/26 00:24:44 ||65cc6da4-9fa6-4e6d-8b69-b99f5d9acacb|audit|127.0.0.1|KNOX|admin|||access|uri|/gateway/admin/api/v1/topologies|success|Response status: 403

VR46 · ‎03-15-2018

Hello @GN_Exp,

There are couple of things here:

1. From your gateway.log (dt:03/12), it looks like the Knox Gateway is trying to initialize RangerPDPKnoxFilter in the Gateway request filter and failing while doing so. This RangerPDPKnoxFilter is used when Kerberos is configured. Since you don't have Kerberos configured, you should not be using this.

2. To enable the Ranger plugin in Knox gateway service, you do not always need XAsecurePDPKnox as authorization provider. "AclsAuthz" would do just fine. This is usually used for Service Level Authorization in Knox topology, which you don't need for Ranger plugin test connection.

Therefore, please stick to "AclsAuthz" unless you have any other use-case. If you still have problem with Knox service repo in Ranger, please attach the screenshot of Knox repo configuration from Ranger UI and screenshot of the error (if any).

Hope this helps !

View solution in original post

dsharma · ‎02-27-2018

@GN_Exp is this cluster an unsecure cluster ? if so the can you please validate the username and password given in the knox repo ?

nhgodwal · ‎03-07-2018

@Deepak Sharma

Yes, the cluster is not Kerberos secured. I have validated the username and corrected the password I was giving while creating service. Now, I am not able to see any error in gateway.log. But gateway-audit.log and ranger-admin.log still showing same error.

Can you suggest?

nhgodwal · ‎03-08-2018

@Deepak Sharma

i am using same username and password as I am using for HDFS service.

dsharma · ‎03-11-2018

GN_Exp is there ranger knox policy present to allow access to admin user the access of ?

nhgodwal · ‎03-11-2018

@Deepak Sharma Thanks for replying.

Yes, when I create knox service, it automatically creates a policy to allow access to admin. But it didn't show in Audit tab. I am attaching screen shots too.

Also, this time I did the fresh intsallation of ranger, hdfs-plugin and then knox plugin. I find logs little different then previous.

gateway-audit.log

18/03/11 22:57:46 ||bf9148a8-fe74-4a1a-b18f-9a300f0a062c|audit|127.0.0.1|KNOX||||access|uri|/gateway/admin/api/v1/topologies|unavailable|Request method: GET
18/03/11 22:57:46 ||bf9148a8-fe74-4a1a-b18f-9a300f0a062c|audit|127.0.0.1|KNOX||||authentication|principal|admin|failure|LDAP authentication failed.
18/03/11 22:57:46 ||bf9148a8-fe74-4a1a-b18f-9a300f0a062c|audit|127.0.0.1|KNOX||||access|uri|/gateway/admin/api/v1/topologies|success|Response status: 401

gateway.log

2018-03-11 22:57:46,092 INFO  hadoop.gateway (KnoxLdapRealm.java:getUserDn(691)) - Computed userDn: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: admin
2018-03-11 22:57:46,118 INFO  hadoop.gateway (KnoxLdapRealm.java:doGetAuthenticationInfo(203)) - Could not login: org.apache.shiro.authc.UsernamePasswordToken - admin, rememberMe=false (127.0.0.1)
2018-03-11 22:57:46,121 ERROR hadoop.gateway (KnoxLdapRealm.java:doGetAuthenticationInfo(205)) - Shiro unable to login: javax.naming.AuthenticationException: [LDAP: error code 49 - INVALID_CREDENTIALS: Bind failed: ERR_229 Cannot authenticate user uid=admin,ou=people,dc=hadoop,dc=apache,dc=org]

ranger_admin.log

2018-03-11 22:57:45,291 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/ranger-knox-plugin-0.7.0.2.6.4.0-91.jar
2018-03-11 22:57:45,291 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/jackson-core-asl-1.9.13.jar
2018-03-11 22:57:45,292 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/commons-collections-3.2.2.jar
2018-03-11 22:57:45,293 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/jackson-mapper-asl-1.9.13.jar
2018-03-11 22:57:45,293 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/commons-lang-2.6.jar
2018-03-11 22:57:46,142 [timed-executor-pool-0] ERROR apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:131) - Got invalid REST response from: https://localhost:8443/gateway/admin/api/v1/topologies, responseStatus: 401

I installed the ranger and plugins referring this link -> https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_command-line-installation/content/ch_ins...

Please let me know if you need me to post any config files.

dsharma · ‎03-12-2018

I think you are using LDAP that comes with knox , can you please check if it is up and running ? and if yes then check for that admin user password in users.ldif file in /etc/knox/conf. by defaut it is admin-password

nhgodwal · ‎03-12-2018

@Deepak Sharma @vsuvagia

yes, I am using demo LDAP comes with knox.

I changed the password from admin to admin-password. Now I am getting different error logs:

Even changing the localhost to fqdn giving me same error logs.

ranger-admin.log

2018-03-12 14:34:55,969 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/ranger-knox-plugin-0.7.0.2.6.4.0-91.jar
2018-03-12 14:34:55,970 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/jackson-core-asl-1.9.13.jar
2018-03-12 14:34:55,970 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/commons-collections-3.2.2.jar
2018-03-12 14:34:55,970 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/jackson-mapper-asl-1.9.13.jar
2018-03-12 14:34:55,970 [http-bio-6080-exec-9] WARN  org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/commons-lang-2.6.jar
2018-03-12 14:34:56,013 [timed-executor-pool-0] ERROR org.apache.ranger.plugin.util.PasswordUtils (PasswordUtils.java:130) - Unable to decrypt password due to error
javax.crypto.IllegalBlockSizeException: Input length must be multiple of 8 when decrypting with padded cipher
        at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:936)
        at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:847)
        at com.sun.crypto.provider.PBES1Core.doFinal(PBES1Core.java:416)
        at com.sun.crypto.provider.PBEWithMD5AndDESCipher.engineDoFinal(PBEWithMD5AndDESCipher.java:316)
        at javax.crypto.Cipher.doFinal(Cipher.java:2165)
        at org.apache.ranger.plugin.util.PasswordUtils.decryptPassword(PasswordUtils.java:115)
        at org.apache.ranger.services.knox.client.KnoxClient.getTopologyList(KnoxClient.java:79)
        at org.apache.ranger.services.knox.client.KnoxClient$2.call(KnoxClient.java:406)
        at org.apache.ranger.services.knox.client.KnoxClient$2.call(KnoxClient.java:402)
        at org.apache.ranger.services.knox.client.KnoxClient.timedTask(KnoxClient.java:431)
        at org.apache.ranger.services.knox.client.KnoxClient.getKnoxResources(KnoxClient.java:410)
        at org.apache.ranger.services.knox.client.KnoxClient.connectionTest(KnoxClient.java:315)
        at org.apache.ranger.services.knox.client.KnoxResourceMgr.validateConfig(KnoxResourceMgr.java:43)
        at org.apache.ranger.services.knox.RangerServiceKnox.validateConfig(RangerServiceKnox.java:56)
        at org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:560)
        at org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:547)
        at org.apache.ranger.biz.ServiceMgr$TimedCallable.call(ServiceMgr.java:508)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
2018-03-12 14:34:56,015 [timed-executor-pool-0] INFO  apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:81) - Password decryption failed; trying knox connection with received password string
2018-03-12 14:34:57,918 [timed-executor-pool-0] ERROR apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:131) - Got invalid REST response from: https://localhost:8443/gateway/admin/api/v1/topologies, responseStatus: 500

gateway-audit.log

18/03/12 14:34:56 ||3e3df102-de59-4c19-9779-cdd4522181bb|audit|127.0.0.1|KNOX||||access|uri|/gateway/admin/api/v1/topologies|unavailable|Request method: GET
18/03/12 14:34:56 ||3e3df102-de59-4c19-9779-cdd4522181bb|audit|127.0.0.1|KNOX|admin|||authentication|uri|/gateway/admin/api/v1/topologies|success|
18/03/12 14:34:56 ||3e3df102-de59-4c19-9779-cdd4522181bb|audit|127.0.0.1|KNOX|admin|||authentication|uri|/gateway/admin/api/v1/topologies|success|Groups: []
18/03/12 14:34:57 ||3e3df102-de59-4c19-9779-cdd4522181bb|audit|127.0.0.1|KNOX|admin|||access|uri|/gateway/admin/api/v1/topologies|failure|

gateway.log

gateway-log.txt