HDP 2.3.4/Ambari 220.127.116.11
I am having some trouble understanding how the various Ranger users are used and where to configure the passwords. What I believe to be true (based on some other posts on Ranger in this forum), is:
Note that I have Ranger configured for Unix sync mode. Ranger admin and usersync processes are running on host api01 and my two HA name nodes are running on host nn01 and nn02.
So, what I am looking for is two things:
03 May 2016 06:04:41 INFO PasswordValidator [Thread-8] - Response [FAILED: [rangerusersync] does not exists.] for user: rangerusersync
03 May 2016 06:04:09 INFO UnixAuthenticationService [main] - Starting User Sync Service! 03 May 2016 06:04:09 INFO UnixAuthenticationService [main] - Enabling Unix Auth Service! 03 May 2016 06:04:09 INFO UserGroupSync [UnixUserSyncThread] - initializing sink: org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder 03 May 2016 06:04:10 WARN NativeCodeLoader [main] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 03 May 2016 06:04:10 INFO UnixAuthenticationService [main] - Enabling Protocol: [SSLv2Hello] 03 May 2016 06:04:10 INFO UnixAuthenticationService [main] - Enabling Protocol: [TLSv1] 03 May 2016 06:04:10 INFO UnixAuthenticationService [main] - Enabling Protocol: [TLSv1.1] 03 May 2016 06:04:10 INFO UnixAuthenticationService [main] - Enabling Protocol: [TLSv1.2] 03 May 2016 06:04:41 INFO PasswordValidator [Thread-8] - Response [FAILED: [rangerusersync] does not exists.] for user: rangerusersync 03 May 2016 06:04:41 ERROR UserGroupSync [UnixUserSyncThread] - Failed to initialize UserGroup source/sink. Will retry after 60000 milliseconds. Error details: com.sun.jersey.api.client.UniformInterfaceException: GET http://api01.qa.quasar.local:6080/service/xusers/groups/?pageSize=1000&startIndex=0 returned a response status of 401 Unauthorized at com.sun.jersey.api.client.WebResource.handle(WebResource.java:686) at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507) at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.buildGroupList(PolicyMgrUserGroupBuilder.java:346) at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.buildUserGroupInfo(PolicyMgrUserGroupBuilder.java:156) at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.init(PolicyMgrUserGroupBuilder.java:152) at org.apache.ranger.usergroupsync.UserGroupSync.run(UserGroupSync.java:51) at java.lang.Thread.run(Thread.java:745)
Thanks in advance...
Hi @Mark Petronic :
What version of Ambari/HDP are you using?
To answer your questions about rangerusersync...
1] rangerusersync is an internal ranger user. See if you can login to Ranger UI using rangerusersync/rangerusersync (default). If this is not working, then you need to see if this password is changed. You can login as admin to verify and reset the password. Once password is changed for rangerusersync user, from usersync install dir (/usr/hdp/current/ranger-usersync) execute "python updatepolicymgrpassword.py" and provide the new password there.
2] It is not required to configure rangerusersync as it is an internal user and there is no requirement that it needs to be running on a particular host.
3] First error is harmless because ranger authentication goes through the normal flow and since you have configured unix authentication and rangerusersync does not exist as OS user, this error is shown and can be ignored. Second error means, usersync process is not able to communicate to ranger admin. See ranger admin logs also to get more clues. This could be because of wrong password (1 above) or any other misconfiguration. Is there an upgrade involved? In that case, make sure /etc/ranger/admin/conf/security-applicationContext.xml matches /usr/hdp/current/ranger-admin/ews/webapp/WEB-INF/classes/conf.dist/security-applicationContext.xml
Thanks, I'm working through this but have a question still regarding my question #2. Can you please explain how usersync obtains the user/groups? Does it read /etc/passwd and /etc/group files from the node that the usersync process is running on or does it use some RPC call to the namenode to obtain that information from those same files that reside on the namenode? I believe that the ONLY place I need to create user and groups is on the namenode (or both namenodes in my case since I am running NN HA). I would just like to understand the process by which Ranger discovers the users and groups so that I can be sure where to create them in the first place. It's a bit of a mystery to me. That's why I asked if usersync should be running on the NNs or can it be running anywhere. You seem to state anywhere is ok.
For question#1: After the password is updated for rangerusersync user, restart ranger in order for the changes to take effect.
Yes, if you are using unix sync, then usersync process needs to be running on the host where you have all the relevant users defined in /etc/passwd.
Ok, I was thinking that must be the case. So, I decided to just remove Ranger and start over with a clean install. I used the Ambari REST APIs to remove the service. Then I dropped all the ranger users and databases from MySQL. However, when I try to reinstall Ranger via Ambari, it will not allow me to install on any node except one - the place where I first installed it - which is NOT the NN where I want to install it. Somehow, I believe Ambari has some left over config that is causing it to lock my choice to just one node. Both Ranger Admin and UserSync are forced to the same node. All the pick lists that you normally use to select a different node to install the service on are all disabled from selection. Any idea how to release Ranger to be installed where I want to?
I spoke to two different HWX support engineers this week who are Ranger experts. I gained a bit more understanding on some points of confusion, so I thought I would pass this on in case anyone has the same questions:
When using Kerberos and running YARN with the LinuxContainerExecutor, it is required to have all users (who wish to run YARN applications) on all nodes!