Member since
08-14-2019
9
Posts
0
Kudos Received
0
Solutions
09-08-2019
03:54 PM
Hi @andrew_ryan1 we got impersonation working using Thrift for users (based on either their user id or groups) when we added property below to HBase site xml: <property> <name>hbase.security.exec.permission.checks</name> <value>true</value> </property> This should work for the REST route as well as it is not specific to REST or Thrift. Needless to say, the real user or their groups need to be given the appropriate privileges (ACLs / visibility tag authorizations). Additionally, these properties added to core-site.xml would enable group lookup for a real user, so groups needed for authorizing a real user can be enumerated (we are yet to test this): <property> <name>hadoop.security.group.mapping</name> <value>org.apache.hadoop.security.LdapGroupsMapping</value> </property> <property> <name>hadoop.security.group.mapping.ldap.url</name> <value>ldap://server</value> </property> <property> <name>hadoop.security.group.mapping.ldap.bind.user</name> <value>Administrator@example-ad.local</value> </property> <property> <name>hadoop.security.group.mapping.ldap.bind.password</name> <value>****</value> </property> <property> <name>hadoop.security.group.mapping.ldap.base</name> <value>dc=example-ad,dc=local</value> </property> <property> <name>hadoop.security.group.mapping.ldap.search.filter.user</name> <value>(&(objectClass=user)(sAMAccountName={0}))</value> </property> <property> <name>hadoop.security.group.mapping.ldap.search.filter.group</name> <value>(objectClass=group)</value> </property> <property> <name>hadoop.security.group.mapping.ldap.search.attr.member</name> <value>member</value> </property> <property> <name>hadoop.security.group.mapping.ldap.search.attr.group.name</name> <value>cn</value> </property>
... View more
09-05-2019
03:16 PM
Hi @elserj while broadly agreeing with the principle of what you are saying, I would amend your earlier comment: "Without enabling Kerberos authentication for HBase, any authorization checks you make are pointless." Knox, in fact offers a HeaderPreAuth Provider for pre-authenticated use cases. It is another matter that it is half baked as far as user groups are concerned. Without belabouring the point any further, impersonation is for users who are not authenticated and so need to piggy back on an authenticated super user. The permissions and ACL still have to be granted to such pre-authenticated users for the resources that they need. It is not the permission and ACL of the super user that is (or should be) utilized for authorization checks for the real user. If the perimeter security provides strong authentication, there should not be a need to further authenticate the same user and too via Krb. Many a resources time is wasted by documentation and commentary recommending this route.
... View more
09-05-2019
01:58 AM
Hi @andrew_ryan1 We gave up on Rest but got proxying working through Thrift (using tHttp Client, then tclient.setCustomHeader("doAs", "realuser") where realuser is the user that the superuser will be proxying in for). We are now scaling the hurdle of getting impersonation working with HBase cell-based security authorization off real user's group memberships. You might also want to try adding these props for the HTTP user to the HDFS core-site.xml: hadoop.proxyuser.HTTP.hosts=* hadoop.proxyuser.HTTP.groups=* hadoop.proxyuser.HTTP.users=* i have looked at the HBase proxy authorization code and it simply looks for these config props for the logged in user (if 'hbase' user, hadoop.proxyuser.hbase.<> props must exist ) and match the values ('*' above) against the values that came in on the client call. Kerberos is not a pre-requisite for this to work.
... View more
09-04-2019
11:45 PM
Hi @elserj That is specious on many counts: - users may have been pre-authenticated. Worse, they could be external partner users having been pre-authenticated in the 'modern' world with OIDC / OAuth. There is no justification (even pleading insanity) to have such users Krb authenticated. In our case, there are 70K+ such users. - the impersonation documentation here that cites the famous user 'joe' states: "The superuser has kerberos credentials but user joe doesn't have any" I thought that is the entire point of proxying / impersonation. In our case, the end user is known, pre-authenticated and if they were not, they can't invoke our HBase query services which are protected behind an API gateway that challenges for a token (jwt).
... View more
09-04-2019
11:30 PM
Hi @andrew_ryan1 as I suspected, suffering is universal in a distributed world... Please ensure you re-start HBase Rest daemon after every config change: ./hbase-daemon.sh [stop|start] rest --port <8090> This, i suspect, is because the Rest daemon refers to the Hbase config. Re-starting affected servers after a config change from Ambari unfortunately leaves out the Rest daemon. As for your problem in the linked post in your reply, when you say all actions are run as root, I'm assuming you mean 'hbase' user? If you literally mean 'root', then it could be the Linux 'root' user may have been used to start the Rest daemon. This happened at my joint 😉
... View more
09-04-2019
05:14 PM
Scenario: Hadoop not kerberized (does not matter even if it were - read below).
We have a setup where users from external partners are authenticated via federated security (OAuth) to access our system (Custom UI, Hadoop/HBase). The groups of the user are granted permissions to HBase tables (R) and cells (HBase cell-level security using visibility tags).
These users do not have a local unix account and no Krb keytab. They are pre-authenticated as above and our system gets their JWT, which contains their group claims, as a consequence.
We have tried HBase impersonation over HBase REST and Thrift. We can pass in the user's id or group and HBase applies the access and visibility controls. HBase does not care what entity (user or group) the doAs represents.
However, when the visibility of cells can be resolved by more than one group of a user, there is no way that HBase impersonation would work in our case.
Going over the HBase config and impersonation documentation, it is clear that impersonation implies that the user to be impersonated either has a local account with group memberships on the OS or has a keytab. Clearly undesirable in our scenario.
I see this as a BIG gap in HBase authorization model. Is there a way out?
... View more
- Tags:
- authorization
- HBase
Labels:
- Labels:
-
Apache HBase
08-20-2019
06:08 PM
Hi @Shelton Thanks for your prompt reply. We tried your suggested changes and it worked! BUT only when the Thrift server is running. Impersonation in a HBase REST setup does not work for us. Essentially, with all the required config and testing in secured as well as unsecured Hadoop and across different versions as above, we have come to the conclusion that the REST doAs functionality is buggy.
... View more
08-20-2019
02:33 AM
Ok. Upgraded to HDP 3.1.0 (HDFS 3.1.1 and HBase 2.0.2). Unsecured setup. Started up HBase Rest daemon (as hbase user) and sent a curl request with ?doAs=realuser. Query returns all data as if executed by 'hbase' superuser as opposed to 'realuser' who does not have authorization to visibility tags attached to some fields. The required config (''hbase.rest.support.proxyuser=true et al as specified in original post) was specified via Ambari on the HBase custom site xml. What am i doing wrong??
... View more
08-19-2019
07:33 PM
Using HDP 3.0.0 bundle which has Hadoop 3.1.0 and HBase 2.0.0.3.0 (Version 2, Alpha release-3; Why??). Ambari is used to manage the setup.
We are trying to achieve user impersonation using Rest proxy in an unsecured and then secured setup. Tried it via code (UGI and doAs() as here, https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Superusers.html, but for HBase). All of these fail and return the data, which has visibility tags attached, for the super user (hbase). That is, all data.
Rest proxy path: curl -> Knox -> HBase REST Gateway -> HBase
In a secured setup, Knox authenticates as knox principal to Gateway, Gateway authenticates to HBase as 'hbase' (tried 'HTTP' user as well) and HBase authenticates as 'hbase'.
The users to be impersonated are ldap authenticated at Knox using the Shiro authentication provider and a default identity assertion provider maps the logged in user to the ?doAs query param passed in the call to HBase Rest Gateway.
The logged in users have 'R' permission granted to the table being queried. One user has auth set for the visibility tags against some fields. Second user is not authorized against the visibility tags.
However, all data is returned from the table always as if the superuser is executing the table query, not the real user impersonated by the superuser as expected.
The required config properties for hadoop.proxyuser and the hbase.security (both authn and authz) are specified in the hbase site xml (tried on the HDFS core site xml as well). Config is matching as specified here -> https://community.cloudera.com/t5/Support-Questions/Hbase-rest-proxying/m-p/150395
Towards the end, i've just left this config in: hbase.rest.support.proxyuser: true. And deleted the rest of config that is required to satisfy the proxy auth check for the superuser. Got no errors. Worked merrily as before. Similarly, i've left half config in and removed others hoping the proxy auth check would fail. Again no effect.
On my eyeballs widening on seeing HBase version 2, Alpha Rel-3 packaged in HDP 3.0.0, i've narrowed down to it being the culprit. I've proposed installing HDP 3.1.0 which has HDFS 3.1.1 and HBase 2.0.2. Will take it for a spin once sys admin spit it out.
My question is why did HDP 3.0.0 bake in an Alpha release of HBase?
Secondly, what else should i look at to make user impersonation work (remember i've tried impersonation via Rest path as well as code in secured as well as unsecured Hadoop)?
... View more
Labels: