We are using HBase as our choice of storage within Hortonworks. We have one node running and are planning to upgrade to multiple nodes if everything works fine. Currently, we use Knox SSO to sign in to the services.
To access the data within HBase we use the WebHBase api. Signing in through Knox SSO works just fine. The user we use to do this ("testuser") has full access rights on Hbase, configured in Ranger.
However, something goes wrong when we are through Knox and arrive at Hbase to retrieve the data. No we get the exception that user "root". How come that it asks for user "root" while we want to fetch data with "testuser"? Obviously we can make a user "root" and give it full clearance, but this is highly undesirable. We think there must be some mistake with the user authorization within Ranger/Knox regarding the services. Does anyone have a clue about this problem? I've been searching for a long time, but no results yet.
Here is the stacktrace we get when accessing the webhbase API through our Knox gateway:
Forbidden org.apache.hadoop.hbase.security.AccessDeniedException: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user ‘root',action: scannerOpen, tableName:testtable, family:r. at org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor.authorizeAccess(RangerAuthorizationCoprocessor.java:511) at org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor.preScannerOpen(RangerAuthorizationCoprocessor.java:901) at org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor.preScannerOpen(RangerAuthorizationCoprocessor.java:856) at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$50.call(RegionCoprocessorHost.java:1267) at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1638) at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1712) at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperationWithResult(RegionCoprocessorHost.java:1687) at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preScannerOpen(RegionCoprocessorHost.java:1262) at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2279) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32295) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2127) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108) at java.lang.Thread.run(Thread.java:745)
"To access the data within HBase we use the WebHBase api" <-- Do you mean the HBase REST server? https://hbase.apache.org/book.html#_rest
This sounds like there's an issue with your Knox setup or the REST tier before the HBase RegionServers. The user that HBase sees is "root" -- this is passed along via the RPC.
You should share your hbase-site.xml, your Knox configuration files, and try to clarify what what's happening before the request reaches HBase.
I'm guessing that "root" is coming from you running a service as the root user on your system. Which, by the way, is a bad idea. My hunch is that Knox is run as root and thus all requests going through Knox appear to be from "root".
You need to configure the HBase REST server to support hbase.rest.support.proxyuser and configure Knox to pass along the real user via the "doAs" parameter in the HTTP query string. I see that you are not using Kerberos, so YMMV with this approach. Having authorization controls without security provides no security, thus it is not something that HBase thoroughly tests. The following docs may help, but they are written expecting that you have Kerberos enabled.
Thanks for the reply. I think we are implementing Kerberos next week. Then I will check how knox is ran within our environment. I will also check the proxyuser settings of hbase.
Many thanks for the new view you gave me.
Thanks for your reply. Yes, I mean the HBase REST server. The REST server itself is working fine, we already checked this. We have configured KnoxSSO. So when you try to access any page or api within the cluster, you will get redirected to the KnoxSSO login page. There you need to give your credentials. This layer works fine and you can login. I will share my hbase-site.xml and knox config files with you below (I changed the domain name and ip address in a generic one to be able to share it).