@Vipin Rathor Thank you so much for the explanation! I have a follow up question too. If I use 'AclsAuthz' provider then I won't be able to do service level authorization in Ranger by creating policies..is this correct? I think because in that case service level authorization will be enforced what I define in knox topology like below under 'AclsAuthz' provider. <param> <name>{serviceName}.acl</name> <value>username[,*|username...];group[,*|group...];ipaddr[,*|ipaddr...]</value> </param> ... View more

@Deepak Sharma @vsuvagia If you look at the gateway.log I attached in my previous post, the main error is -> ERROR knox.RangerPDPKnoxFilter (RangerPDPKnoxFilter.java:init(73)) - Error while setting UGI for Knox Plugin... I tried to look for this error and I found this post -> https://community.hortonworks.com/questions/97518/help-ad-integration-with-knox.html In this post, the resolution is to change Authorization provider in admin topology from XAsecurePDPKnox to AclsAuthz. I tried that too and I am getting a successfull connection by changing this but I read somewhere that to enable Ranger plugin, authorization provider has to be XAsecurePDPKnox. Please suggest. ... View more

@vsuvagia Yes, I did import the knox cert to Ranger truststore. I am not getting any SSL error. ... View more

@Deepak Sharma @vsuvagia yes, I am using demo LDAP comes with knox. I changed the password from admin to admin-password. Now I am getting different error logs: Even changing the localhost to fqdn giving me same error logs. ranger-admin.log 2018-03-12 14:34:55,969 [http-bio-6080-exec-9] WARN org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/ranger-knox-plugin-0.7.0.2.6.4.0-91.jar 2018-03-12 14:34:55,970 [http-bio-6080-exec-9] WARN org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/jackson-core-asl-1.9.13.jar 2018-03-12 14:34:55,970 [http-bio-6080-exec-9] WARN org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/commons-collections-3.2.2.jar 2018-03-12 14:34:55,970 [http-bio-6080-exec-9] WARN org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/jackson-mapper-asl-1.9.13.jar 2018-03-12 14:34:55,970 [http-bio-6080-exec-9] WARN org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/commons-lang-2.6.jar 2018-03-12 14:34:56,013 [timed-executor-pool-0] ERROR org.apache.ranger.plugin.util.PasswordUtils (PasswordUtils.java:130) - Unable to decrypt password due to error javax.crypto.IllegalBlockSizeException: Input length must be multiple of 8 when decrypting with padded cipher at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:936) at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:847) at com.sun.crypto.provider.PBES1Core.doFinal(PBES1Core.java:416) at com.sun.crypto.provider.PBEWithMD5AndDESCipher.engineDoFinal(PBEWithMD5AndDESCipher.java:316) at javax.crypto.Cipher.doFinal(Cipher.java:2165) at org.apache.ranger.plugin.util.PasswordUtils.decryptPassword(PasswordUtils.java:115) at org.apache.ranger.services.knox.client.KnoxClient.getTopologyList(KnoxClient.java:79) at org.apache.ranger.services.knox.client.KnoxClient$2.call(KnoxClient.java:406) at org.apache.ranger.services.knox.client.KnoxClient$2.call(KnoxClient.java:402) at org.apache.ranger.services.knox.client.KnoxClient.timedTask(KnoxClient.java:431) at org.apache.ranger.services.knox.client.KnoxClient.getKnoxResources(KnoxClient.java:410) at org.apache.ranger.services.knox.client.KnoxClient.connectionTest(KnoxClient.java:315) at org.apache.ranger.services.knox.client.KnoxResourceMgr.validateConfig(KnoxResourceMgr.java:43) at org.apache.ranger.services.knox.RangerServiceKnox.validateConfig(RangerServiceKnox.java:56) at org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:560) at org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:547) at org.apache.ranger.biz.ServiceMgr$TimedCallable.call(ServiceMgr.java:508) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2018-03-12 14:34:56,015 [timed-executor-pool-0] INFO apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:81) - Password decryption failed; trying knox connection with received password string 2018-03-12 14:34:57,918 [timed-executor-pool-0] ERROR apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:131) - Got invalid REST response from: https://localhost:8443/gateway/admin/api/v1/topologies, responseStatus: 500 gateway-audit.log 18/03/12 14:34:56 ||3e3df102-de59-4c19-9779-cdd4522181bb|audit|127.0.0.1|KNOX||||access|uri|/gateway/admin/api/v1/topologies|unavailable|Request method: GET 18/03/12 14:34:56 ||3e3df102-de59-4c19-9779-cdd4522181bb|audit|127.0.0.1|KNOX|admin|||authentication|uri|/gateway/admin/api/v1/topologies|success| 18/03/12 14:34:56 ||3e3df102-de59-4c19-9779-cdd4522181bb|audit|127.0.0.1|KNOX|admin|||authentication|uri|/gateway/admin/api/v1/topologies|success|Groups: [] 18/03/12 14:34:57 ||3e3df102-de59-4c19-9779-cdd4522181bb|audit|127.0.0.1|KNOX|admin|||access|uri|/gateway/admin/api/v1/topologies|failure| gateway.log gateway-log.txt ... View more

@Deepak Sharma Thanks for replying. Yes, when I create knox service, it automatically creates a policy to allow access to admin. But it didn't show in Audit tab. I am attaching screen shots too. Also, this time I did the fresh intsallation of ranger, hdfs-plugin and then knox plugin. I find logs little different then previous. gateway-audit.log 18/03/11 22:57:46 ||bf9148a8-fe74-4a1a-b18f-9a300f0a062c|audit|127.0.0.1|KNOX||||access|uri|/gateway/admin/api/v1/topologies|unavailable|Request method: GET 18/03/11 22:57:46 ||bf9148a8-fe74-4a1a-b18f-9a300f0a062c|audit|127.0.0.1|KNOX||||authentication|principal|admin|failure|LDAP authentication failed. 18/03/11 22:57:46 ||bf9148a8-fe74-4a1a-b18f-9a300f0a062c|audit|127.0.0.1|KNOX||||access|uri|/gateway/admin/api/v1/topologies|success|Response status: 401 gateway.log 2018-03-11 22:57:46,092 INFO hadoop.gateway (KnoxLdapRealm.java:getUserDn(691)) - Computed userDn: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: admin 2018-03-11 22:57:46,118 INFO hadoop.gateway (KnoxLdapRealm.java:doGetAuthenticationInfo(203)) - Could not login: org.apache.shiro.authc.UsernamePasswordToken - admin, rememberMe=false (127.0.0.1) 2018-03-11 22:57:46,121 ERROR hadoop.gateway (KnoxLdapRealm.java:doGetAuthenticationInfo(205)) - Shiro unable to login: javax.naming.AuthenticationException: [LDAP: error code 49 - INVALID_CREDENTIALS: Bind failed: ERR_229 Cannot authenticate user uid=admin,ou=people,dc=hadoop,dc=apache,dc=org] ranger_admin.log 2018-03-11 22:57:45,291 [http-bio-6080-exec-9] WARN org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/ranger-knox-plugin-0.7.0.2.6.4.0-91.jar 2018-03-11 22:57:45,291 [http-bio-6080-exec-9] WARN org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/jackson-core-asl-1.9.13.jar 2018-03-11 22:57:45,292 [http-bio-6080-exec-9] WARN org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/commons-collections-3.2.2.jar 2018-03-11 22:57:45,293 [http-bio-6080-exec-9] WARN org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/jackson-mapper-asl-1.9.13.jar 2018-03-11 22:57:45,293 [http-bio-6080-exec-9] WARN org.apache.ranger.biz.ServiceMgr (ServiceMgr.java:355) - getFilesInDirectory('ranger-plugins/knox'): adding /usr/hdp/2.6.4.0-91/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/knox/commons-lang-2.6.jar 2018-03-11 22:57:46,142 [timed-executor-pool-0] ERROR apache.ranger.services.knox.client.KnoxClient (KnoxClient.java:131) - Got invalid REST response from: https://localhost:8443/gateway/admin/api/v1/topologies, responseStatus: 401 I installed the ranger and plugins referring this link -> https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_command-line-installation/content/ch_installing_ranger_chapter.html Please let me know if you need me to post any config files. ... View more

@Deepak Sharma i am using same username and password as I am using for HDFS service. ... View more

@Deepak Sharma Yes, the cluster is not Kerberos secured. I have validated the username and corrected the password I was giving while creating service. Now, I am not able to see any error in gateway.log. But gateway-audit.log and ranger-admin.log still showing same error. Can you suggest? ... View more

Thanks @spolavarapu. This worked for me. ... View more

@spolavarapu Thanks for the clarification. Can you please tell me how to disable default incremental sync. I am doing manual installation (not with Ambari). I am not sure which property I need to set for disabling incremental sync. ... View more