Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

User impersonation fails in secured HBase

User impersonation fails in secured HBase

Explorer

Using HDP 3.0.0 bundle which has Hadoop 3.1.0 and HBase 2.0.0.3.0 (Version 2, Alpha release-3; Why??). Ambari is used to manage the setup.

 

We are trying to achieve user impersonation using Rest proxy in an unsecured and then secured setup. Tried it via code (UGI and doAs() as here, https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Superusers.html, but for HBase). All of these fail and return the data, which has visibility tags attached, for the super user (hbase). That is, all data.

 

Rest proxy path: curl -> Knox -> HBase REST Gateway -> HBase

 

In a secured setup, Knox authenticates as knox principal to Gateway, Gateway authenticates to HBase as 'hbase' (tried 'HTTP' user as well) and HBase authenticates as 'hbase'.

 

The users to be impersonated are ldap authenticated at Knox using the Shiro authentication provider and a default identity assertion provider maps the logged in user to the ?doAs query param passed in the call to HBase Rest Gateway.

 

The logged in users have 'R' permission granted to the table being queried. One user has auth set for the visibility tags against some fields. Second user is not authorized against the visibility tags.

 

However, all data is returned from the table always as if the superuser is executing the table query, not the real user impersonated by the superuser as expected.

 

The required config properties for hadoop.proxyuser and the hbase.security (both authn and authz) are specified in the hbase site xml (tried on the HDFS core site xml as well). Config is matching as specified here -> https://community.cloudera.com/t5/Support-Questions/Hbase-rest-proxying/m-p/150395

 

Towards the end, i've just left this config in: hbase.rest.support.proxyuser: true. And deleted the rest of config that is required to satisfy the proxy auth check for the superuser. Got no errors. Worked merrily as before. Similarly, i've left half config in and removed others hoping the proxy auth check would fail. Again no effect.

 

On my eyeballs widening on seeing HBase version 2, Alpha Rel-3 packaged in HDP 3.0.0, i've narrowed down to it being the culprit. I've proposed installing HDP 3.1.0 which has HDFS 3.1.1 and HBase 2.0.2. Will take it for a spin once sys admin spit it out.

 

My question is why did HDP 3.0.0 bake in an Alpha release of HBase?

 

Secondly, what else should i look at to make user impersonation work (remember i've tried impersonation via Rest path as well as code in secured as well as unsecured Hadoop)?

10 REPLIES 10

Re: User impersonation fails in secured HBase

Explorer

Ok. Upgraded to HDP 3.1.0 (HDFS 3.1.1 and HBase 2.0.2). Unsecured setup. Started up HBase Rest daemon (as hbase user) and sent a curl request with ?doAs=realuser. Query returns all data as if executed by 'hbase' superuser as opposed to 'realuser' who does not have authorization to visibility tags attached to some fields.

 

The required config (''hbase.rest.support.proxyuser=true et al as specified in original post) was specified via Ambari on the HBase custom site xml.

 

What am i doing wrong??

Highlighted

Re: User impersonation fails in secured HBase

Mentor

Make sure you have changed these properties in your hbase-site.xml:


<property>
<name>hbase.thrift.support.proxyuser</name>
<value>true</value>
</property>

<property>
<name>hbase.regionserver.thrift.http</name>
<value>true</value>
</property>


Then check in HDFS-->Configs-->Advanced-Custome core-site.xml add these 2 properties to enable HBase to impersonates a user

 

 <property>
<name>hadoop.proxyuser.hbase.hosts</name>
<value>*</value>
</property>

<property>
<name>hadoop.proxyuser.hbase.groups</name>
<value>*</value>
</property>                       

Please do these modifications and start any stale configs and revert

Re: User impersonation fails in secured HBase

Explorer

Hi @Shelton Thanks for your prompt reply.

 

We tried your suggested changes and it worked! BUT only when the Thrift server is running.

 

Impersonation in a HBase REST setup does not work for us. Essentially, with all the required config and testing in secured as well as unsecured Hadoop and across different versions as above, we have come to the conclusion that the REST doAs functionality is buggy.

Re: User impersonation fails in secured HBase

Contributor

I'm having a very similar issue.

I posted this on the 08-08-2019

https://community.cloudera.com/t5/Support-Questions/Ranger-hbase-plugin-not-proxying-users-Runs-ever...

HDP 3.0.1 hbase 2.0.0.3.0.1.0-187

Knox is dispatching the request to hbase correctly.

19/09/05 14:39:31 ||c5231017-9998-458d-9336-98721bcb7cb2|audit|39.7.48.21|WEBHBASE|c8ary|||dispatch|uri|https://<namenode>:60080/footy/1?doAs=andrew|unavailable|Request method: GET
19/09/05 14:39:31 ||c5231017-9998-458d-9336-98721bcb7cb2|audit|39.7.48.21|WEBHBASE|c8ary|||dispatch|uri|https://<namenode>:60080/footy/1?doAs=andrew|success|Response status: 200
19/09/05 14:39:31 |||audit|39.7.48.21|WEBHBASE|andrew|||access|uri|/gateway/default/hbase/footy/1|success|Response status: 200

 

Every request is run as HTTP (which I have specified as the rest principal and rest authentication principal)

hbase.rest.authentication.kerberos.keytab
/etc/security/keytabs/spnego.service.keytab
hbase.rest.authentication.kerberos.principal
HTTP/_HOST@REALM
hbase.rest.kerberos.principal
HTTP/_HOST@REALM
hbase.rest.keytab.file
/etc/security/keytabs/spnego.service.keytab

hbase-ranger-plugin is working. When I remove the HTTP user from the ranger policy access is denied. 

doAs functionality seems buggy to me also.

Re: User impersonation fails in secured HBase

Explorer

Hi @andrew_ryan1 

 

as I suspected, suffering is universal in a distributed world...

 

Please ensure you re-start HBase Rest daemon after every config change:

 

./hbase-daemon.sh [stop|start] rest --port <8090>

 

This, i suspect, is because the Rest daemon refers to the Hbase config. Re-starting affected servers after a config change from Ambari unfortunately leaves out the Rest daemon.

 

As for your problem in the linked post in your reply, when you say all actions are run as root, I'm assuming you mean 'hbase' user? If you literally mean 'root', then it could be the Linux 'root' user may have been used to start the Rest daemon. This happened at my joint

Re: User impersonation fails in secured HBase

Contributor

Hi @hbased

Did you get it working?

Yeah, I was running the rest server as root with no Kerberos enabled. Curl requests through Knox were all run as root.

Now with Kerberos enabled and running the rest server as hbase rather than root every request is run as HTTP. Which is my proxyuser. 

I have been restarting the rest server but will try again tomorrow as a sanity check.

 

Cheers

Re: User impersonation fails in secured HBase

Explorer

Hi @andrew_ryan1 

 

We gave up on Rest but got proxying working through Thrift (using tHttp Client, then tclient.setCustomHeader("doAs", "realuser") where realuser is the user that the superuser will be proxying in for).

 

We are now scaling the hurdle of getting impersonation working with HBase cell-based security authorization off real user's group memberships.

 

You might also want to try adding these props for the HTTP user to the HDFS core-site.xml:

 

hadoop.proxyuser.HTTP.hosts=*

hadoop.proxyuser.HTTP.groups=*

hadoop.proxyuser.HTTP.users=*

 

i have looked at the HBase proxy authorization code and it simply looks for these config props for the logged in user (if 'hbase' user, hadoop.proxyuser.hbase.<> props must exist) and match the values ('*' above) against the values that came in on the client call. Kerberos is not a pre-requisite for this to work.

 

 

Re: User impersonation fails in secured HBase

New Contributor

Hi,

 

Did you also checked in the Knox configuration if the port for WEBHBASE is matching the port from hbase rest?

Re: User impersonation fails in secured HBase

Contributor

Hi @hbased 

I restarted my hbase rest servers - no joy.

I have the hadoop.proxyuser.HTTP.<> settings in core site.

I think I'll have a trawl through the hbase code and raise a jira.

Good luck with the cell based auth.

Thanks for your help!

Cheers

Andrew