HDP and Kerbrose UserSync problems

[#1]

Just deploy the HDP on CentOS through Ambari (2.6.0.0) and then enabled Kerbrose. All seems to be set until i start setting the Ranger policies for users. I think earlier before kerbrose was enabled, ranger was able to sync users with Unix users. I can see the list of users when i am creating a new policy but after Kerbrose was set only old users are being shown in the list (no new Linux users)

Here is the error that i can see in /var/log/ranger/usersync/usersync.log

> 08 May 2018 08:33:19 ERROR PolicyMgrUserGroupBuilder [UnixUserSyncThread] - Failed to add portal user
08 May 2018 08:33:19 ERROR UnixUserGroupBuilder [UnixUserSyncThread] - sink.addOrUpdateUser failed with exception: Failed to add portal user, for user: mapred, groups: [hadoop]
08 May 2018 08:33:19 ERROR PolicyMgrUserGroupBuilder [UnixUserSyncThread] - Failed to add User : 
com.sun.jersey.api.client.UniformInterfaceException: POST http://192.168.99.101:6080/service/users/default returned a response status of 401 Unauthorized
    at com.sun.jersey.api.client.WebResource.handle(WebResource.java:686)
    at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
    at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:568)
    at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.getMUser(PolicyMgrUserGroupBuilder.java:963)
    at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.access$800(PolicyMgrUserGroupBuilder.java:74)
    at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder$5.run(PolicyMgrUserGroupBuilder.java:936)
    at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder$5.run(PolicyMgrUserGroupBuilder.java:932)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:360)
    at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.addMUser(PolicyMgrUserGroupBuilder.java:932)
    at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.addOrUpdateUser(PolicyMgrUserGroupBuilder.java:329)
    at org.apache.ranger.unixusersync.process.UnixUserGroupBuilder.updateSink(UnixUserGroupBuilder.java:153)
    at org.apache.ranger.usergroupsync.UserGroupSync.syncUserGroup(UserGroupSync.java:114)
    at org.apache.ranger.usergroupsync.UserGroupSync.run(UserGroupSync.java:87)
    at java.lang.Thread.run(Thread.java:745)
08 May 2018 08:33:19 ERROR PolicyMgrUserGroupBuilder [UnixUserSyncThread] - Failed to add portal user
08 May 2018 08:33:19 ERROR UnixUserGroupBuilder [UnixUserSyncThread] - sink.addOrUpdateUser failed with exception: Failed to add portal user, for user: knox, groups: [hadoop]
08 May 2018 08:33:19  INFO UserGroupSync [UnixUserSyncThread] - End: update user/group from source==>sink

There is now a question

After setting the Kerbrose and understanding how it works now i am confused between Kerbrose and Unix users. What is the relationship between Unix and Kerbrose users. Yes i can get tickets from kerbrose using kinit and can verify that ticker with klist BUT how does it play in Hadoop. Do i just need Kerbrose user to do i need both Kerbrose and Unix users to access resources in Ranger. I though after enabling Kerbrose then only Kerbrose users will be able to access resources in HDP. Notice i am not using AD or Active Direcoty just experimenting with Kerbrose and Unix

[#2]

This error means Ranger usersync is not able to communicate to Ranger admin to sync the users. Usersync uses kerberos to communicate to Ranger Admin.

Make sure you have all the right keytabs for ranger components (if not, regenerate keytabs from Ambari) and the keytab files have right principals and permissions.

Check for any errors in both usersync logs and ranger admin logs (are there any errors related to spnego keytab?)

[#3]

If you deployed Knox after enabling Kerberos, you will need to recreate keytabs from Ambari > Security

Ranger will only sync from one provider at a time UNIX or LDAP/AD. When you switch to LDAP/AD, new users in UNIX will not be synced. So if you added Knox after sync, knox user will not be synced.

The second part of your question, enabling kerberos does not, just, enable users to access cluster resources. This process actually creates principals and keytabs for all HDP service accounts. This is totally separate from user access. If you need just user access with no security you can do with just LDAP, no kerberos (not recommended on production systems).

[#4]

@Shahzad Aslam

Have you toggled your sync source in Ranger UI to LDAP/AD? see attached screenshot.

Question

Unix users are local users you can also kinit to use the kerberized service. What do you mean by Kerberos users? Your LDAP users should also be able to execute any job on the cluster provided they grab a valid Kerberos ticket.

$ kinit jerry@EXAMPLE.COM 
Password for jerry@EXAMPLE.COM:

It should work correctly, please revert

[#5]

@Shahzad Aslam

Any updates?